[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:56:31 dist-test-slave-zkpd krb5kdc[24915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:56:31 dist-test-slave-zkpd krb5kdc[24915](info): set up 2 sockets
Aug 06 12:56:31 dist-test-slave-zkpd krb5kdc[24915](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 12:56:32.399678 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:56:32 dist-test-slave-zkpd krb5kdc[24915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:56:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:56:32.518642 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:41261
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:41261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:32.621716 24928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:32.622100 24928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:32.622205 24928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:32.623497 24928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:32.623584 24928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:32.623608 24928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:32.623631 24928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:32.623659 24928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:32.625504 24928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:41261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:41261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:32.625921 24928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:32.626222 24928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:32.632997 24934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:32.633159 24937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:32.633015 24933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:32.633742 24928 server_base.cc:1034] running on GCE node
I20260806 12:56:32.634060 24928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:32.634431 24928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:32.635598 24928 hybrid_clock.cc:648] HybridClock initialized: now 1786020992635584 us; error 61 us; skew 500 ppm
Aug 06 12:56:32 dist-test-slave-zkpd krb5kdc[24915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020992, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:32.638162 24928 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:56:32.638665 24928 webserver.cc:460] Webserver started at http://127.24.83.62:33401/ using document root <none> and password file <none>
I20260806 12:56:32.638938 24928 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:32.638983 24928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:32.639098 24928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:32.640280 24928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "123ab7bcbee44cae81eac6cef7e43b83"
format_stamp: "Formatted at 2026-08-06 12:56:32 on dist-test-slave-zkpd"
server_key: "bca97ff9263c271a4b2a1889f62e31ed"
server_key_iv: "41b038425c70980a8a0a7f798a024b5d"
server_key_version: "encryptionkey@0"
I20260806 12:56:32.640741 24928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "123ab7bcbee44cae81eac6cef7e43b83"
format_stamp: "Formatted at 2026-08-06 12:56:32 on dist-test-slave-zkpd"
server_key: "bca97ff9263c271a4b2a1889f62e31ed"
server_key_iv: "41b038425c70980a8a0a7f798a024b5d"
server_key_version: "encryptionkey@0"
I20260806 12:56:32.642439 24928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:56:32.643383 24944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:32.643707 24928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:32.643806 24928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "123ab7bcbee44cae81eac6cef7e43b83"
format_stamp: "Formatted at 2026-08-06 12:56:32 on dist-test-slave-zkpd"
server_key: "bca97ff9263c271a4b2a1889f62e31ed"
server_key_iv: "41b038425c70980a8a0a7f798a024b5d"
server_key_version: "encryptionkey@0"
I20260806 12:56:32.643878 24928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:32.657323 24928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:32.659358 24928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:32.659674 24928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:32.664422 24928 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:41261
I20260806 12:56:32.664477 24996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:41261 every 8 connection(s)
I20260806 12:56:32.665014 24928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:32.666261 24997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:32.669204 24997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83: Bootstrap starting.
I20260806 12:56:32.670122 24997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:32.670728 24997 log.cc:824] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:32.671502 24997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83: No bootstrap required, opened a new log
I20260806 12:56:32.674302 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 24928
I20260806 12:56:32.674427 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:32.674711 24997 raft_consensus.cc:348] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "123ab7bcbee44cae81eac6cef7e43b83" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41261 } }
I20260806 12:56:32.674845 24997 raft_consensus.cc:374] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:32.674916 24997 raft_consensus.cc:729] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 123ab7bcbee44cae81eac6cef7e43b83, State: Initialized, Role: FOLLOWER
I20260806 12:56:32.675105 24908 external_mini_cluster.cc:1428] Setting key 968355d30c160d30610032a3dc041bc7
I20260806 12:56:32.675128 24997 consensus_queue.cc:260] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "123ab7bcbee44cae81eac6cef7e43b83" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41261 } }
I20260806 12:56:32.675280 24997 raft_consensus.cc:388] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:32.675345 24997 raft_consensus.cc:482] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:32.675413 24997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:32.676204 24997 raft_consensus.cc:504] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "123ab7bcbee44cae81eac6cef7e43b83" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41261 } }
I20260806 12:56:32.676342 24997 leader_election.cc:302] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 123ab7bcbee44cae81eac6cef7e43b83; no voters:
I20260806 12:56:32.676582 24997 leader_election.cc:288] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:32.676623 25000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:32.676833 25000 raft_consensus.cc:686] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [term 1 LEADER]: Becoming Leader. State: Replica: 123ab7bcbee44cae81eac6cef7e43b83, State: Running, Role: LEADER
I20260806 12:56:32.676975 25000 consensus_queue.cc:237] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "123ab7bcbee44cae81eac6cef7e43b83" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41261 } }
I20260806 12:56:32.677131 24997 sys_catalog.cc:564] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:32.677448 25000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "123ab7bcbee44cae81eac6cef7e43b83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "123ab7bcbee44cae81eac6cef7e43b83" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41261 } } }
I20260806 12:56:32.677532 25000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:32.677659 25000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 123ab7bcbee44cae81eac6cef7e43b83. Latest consensus state: current_term: 1 leader_uuid: "123ab7bcbee44cae81eac6cef7e43b83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "123ab7bcbee44cae81eac6cef7e43b83" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41261 } } }
I20260806 12:56:32.677745 25000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83 [sys.catalog]: This master's current role is: LEADER
W20260806 12:56:32.681109 25018 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:56:32.681195 25018 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:56:32.681345 25006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:32.681829 25006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:32.686105 25006 catalog_manager.cc:1347] Generated new cluster ID: b3f4969fc4fb4eecb57bbb95feab1975
I20260806 12:56:32.686177 25006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:56:32 dist-test-slave-zkpd krb5kdc[24915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:32.694909 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:32.676527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34226 (local address 127.24.83.62:41261)
0806 12:56:32.677020 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:32.677030 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:32.678215 (+ 1185us) server_negotiation.cc:407] Connection header received
0806 12:56:32.681510 (+ 3295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:32.681516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:32.681629 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:32.681779 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:32.683506 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:32.685538 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:32.686760 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:32.687227 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:32.687364 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:32.691098 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:32.691108 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:32.691109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:32.691118 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:32.692867 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:32.693324 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:32.693327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:32.693328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:32.693433 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:32.693719 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:32.693720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:32.693721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:32.693969 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:32.694146 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:32.694418 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 12:56:32.694603 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":79,"threads_started":1}
I20260806 12:56:32.695992 25006 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:32.696717 25006 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:32.711628 25006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 123ab7bcbee44cae81eac6cef7e43b83: Generated new TSK 0
I20260806 12:56:32.711803 25006 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:56:32.743453 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41261
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:32.852492 25023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:32.852741 25023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:32.852782 25023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:32.853981 25023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:32.854046 25023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:32.854094 25023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:56:32.855721 25023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41261
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:32.856029 25023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:32.856271 25023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:32.862776 25030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:32.862964 25032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:32.862802 25029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:32.863226 25023 server_base.cc:1034] running on GCE node
I20260806 12:56:32.863427 25023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:32.863698 25023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:32.864851 25023 hybrid_clock.cc:648] HybridClock initialized: now 1786020992864833 us; error 35 us; skew 500 ppm
Aug 06 12:56:32 dist-test-slave-zkpd krb5kdc[24915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020992, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:32.871882 25023 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:56:32.872398 25023 webserver.cc:460] Webserver started at http://127.24.83.1:42877/ using document root <none> and password file <none>
I20260806 12:56:32.872624 25023 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:32.872670 25023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:32.872812 25023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:32.873929 25023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "8faa2ae83e1048f6adec9bfc5765af41"
format_stamp: "Formatted at 2026-08-06 12:56:32 on dist-test-slave-zkpd"
server_key: "5a27022ae69508b0ed6a1ea37acc86bc"
server_key_iv: "5798a7a463a6205416b048ddb02be729"
server_key_version: "encryptionkey@0"
I20260806 12:56:32.874296 25023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "8faa2ae83e1048f6adec9bfc5765af41"
format_stamp: "Formatted at 2026-08-06 12:56:32 on dist-test-slave-zkpd"
server_key: "5a27022ae69508b0ed6a1ea37acc86bc"
server_key_iv: "5798a7a463a6205416b048ddb02be729"
server_key_version: "encryptionkey@0"
I20260806 12:56:32.876041 25023 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260806 12:56:32.877022 25039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:32.877241 25023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:32.877332 25023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "8faa2ae83e1048f6adec9bfc5765af41"
format_stamp: "Formatted at 2026-08-06 12:56:32 on dist-test-slave-zkpd"
server_key: "5a27022ae69508b0ed6a1ea37acc86bc"
server_key_iv: "5798a7a463a6205416b048ddb02be729"
server_key_version: "encryptionkey@0"
I20260806 12:56:32.877388 25023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:32.915860 25023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:32.917893 25023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:32.918017 25023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:32.918262 25023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:32.918655 25023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:32.918726 25023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:32.918772 25023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:32.918807 25023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:32.929869 25023 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:44637
I20260806 12:56:32.929972 25159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:44637 every 8 connection(s)
I20260806 12:56:32.930296 25023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:56:32.930613 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 25023
I20260806 12:56:32.930720 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:56:32.930895 24908 external_mini_cluster.cc:1428] Setting key 700d2800ccbf229ac740348950e6ac96
Aug 06 12:56:32 dist-test-slave-zkpd krb5kdc[24915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020992, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
I20260806 12:56:32.941881 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:32.931168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:36899 (local address 127.24.83.62:41261)
0806 12:56:32.931216 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:32.931218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:32.931592 (+ 374us) server_negotiation.cc:407] Connection header received
0806 12:56:32.932368 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:32.932370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:32.932386 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:32.932432 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:32.934134 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:32.934852 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:32.935933 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:32.936254 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:32.936346 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:32.938831 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:32.938833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:32.938833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:32.938841 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:32.940579 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:32.940957 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:32.940958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:32.940958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:32.941020 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:32.941246 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:32.941247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:32.941248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:32.941426 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:32.941510 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:32.941749 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 12:56:32.941817 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:56:32.942077 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:32.931161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:41261 (local address 127.24.83.1:36899)
0806 12:56:32.931418 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:32.931454 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:32.932256 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:32.932509 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:32.932513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:32.933571 (+ 1058us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:32.934015 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:32.934023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:32.934949 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:32.934950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:32.935820 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:32.935823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:32.936351 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:32.936352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:32.936495 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:32.937072 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:32.937079 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:32.938730 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:32.940659 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:32.940665 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:32.940666 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:32.940870 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:32.941086 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:32.941087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:32.941087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:32.941177 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:32.941498 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:32.941507 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:32.941623 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 12:56:32.941708 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 12:56:32.941826 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":128,"threads_started":1}
I20260806 12:56:32.942638 25160 heartbeater.cc:347] Connected to a master server at 127.24.83.62:41261
I20260806 12:56:32.942732 25160 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:32.942963 25160 heartbeater.cc:511] Master 127.24.83.62:41261 requested a full tablet report, sending...
I20260806 12:56:32.943833 24961 ts_manager.cc:194] Registered new tserver with Master: 8faa2ae83e1048f6adec9bfc5765af41 (127.24.83.1:44637)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
I20260806 12:56:32.944800 24961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:36899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:56:32.958854 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41261
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:33.054476 25165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:33.054674 25165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:33.054698 25165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:33.055795 25165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:33.055827 25165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:33.055851 25165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:56:33.057389 25165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41261
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:33.057566 25165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:33.057734 25165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:33.063128 25173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:33.063264 25170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:33.063264 25171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:33.063427 25165 server_base.cc:1034] running on GCE node
I20260806 12:56:33.063664 25165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:33.063848 25165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:33.065033 25165 hybrid_clock.cc:648] HybridClock initialized: now 1786020993065022 us; error 27 us; skew 500 ppm
Aug 06 12:56:33 dist-test-slave-zkpd krb5kdc[24915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020993, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:33.067750 25165 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:56:33.068245 25165 webserver.cc:460] Webserver started at http://127.24.83.2:41933/ using document root <none> and password file <none>
I20260806 12:56:33.068472 25165 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:33.068540 25165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:33.068688 25165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:33.069761 25165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "a7740acac67b46cba36b635049f143e9"
format_stamp: "Formatted at 2026-08-06 12:56:33 on dist-test-slave-zkpd"
server_key: "b6d1f1ed3f5681acfb793e527d97b8cd"
server_key_iv: "0f8c6db33a0f9cf0198baf2b6510d524"
server_key_version: "encryptionkey@0"
I20260806 12:56:33.070140 25165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "a7740acac67b46cba36b635049f143e9"
format_stamp: "Formatted at 2026-08-06 12:56:33 on dist-test-slave-zkpd"
server_key: "b6d1f1ed3f5681acfb793e527d97b8cd"
server_key_iv: "0f8c6db33a0f9cf0198baf2b6510d524"
server_key_version: "encryptionkey@0"
I20260806 12:56:33.071581 25165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:56:33.072463 25181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:33.072649 25165 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 12:56:33.072747 25165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "a7740acac67b46cba36b635049f143e9"
format_stamp: "Formatted at 2026-08-06 12:56:33 on dist-test-slave-zkpd"
server_key: "b6d1f1ed3f5681acfb793e527d97b8cd"
server_key_iv: "0f8c6db33a0f9cf0198baf2b6510d524"
server_key_version: "encryptionkey@0"
I20260806 12:56:33.072827 25165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:33.115212 25165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:33.117329 25165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:33.117551 25165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:33.117901 25165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:33.118238 25165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:33.118294 25165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:33.118327 25165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:33.118374 25165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:33.128626 25165 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:39881
I20260806 12:56:33.128671 25297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:39881 every 8 connection(s)
I20260806 12:56:33.128990 25165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:56:33.134479 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 25165
I20260806 12:56:33.134575 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:56:33.134728 24908 external_mini_cluster.cc:1428] Setting key 9cfbdbc7157cab86d153147857bd92e7
Aug 06 12:56:33 dist-test-slave-zkpd krb5kdc[24915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020993, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:33.139726 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:33.129783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:55931 (local address 127.24.83.62:41261)
0806 12:56:33.129849 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:33.129851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:33.130215 (+ 364us) server_negotiation.cc:407] Connection header received
0806 12:56:33.130940 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:33.130941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:33.130951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:33.131004 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:33.132068 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.132558 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.133651 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.133847 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.133904 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:33.136585 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:33.136587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:33.136587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:33.136595 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:33.138226 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:33.138781 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:33.138781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:33.138782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:33.138830 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:33.139114 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:33.139115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:33.139115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:33.139209 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:33.139264 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:33.139602 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 12:56:33.139664 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 12:56:33.140112 25300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:33.129754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:41261 (local address 127.24.83.2:55931)
0806 12:56:33.129982 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:33.130020 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:33.130849 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:33.131062 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:33.131065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:33.131719 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:33.131984 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:33.131991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.132638 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.132639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:33.133576 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:33.133578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.133919 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.133920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:33.134062 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:33.134719 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:33.134728 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:33.136496 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:33.138321 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:33.138329 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:33.138332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:33.138691 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:33.138894 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:33.138895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:33.138895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:33.139043 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:33.139279 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:33.139285 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:33.139454 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 12:56:33.139585 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 12:56:33.139763 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":121,"threads_started":1}
I20260806 12:56:33.140651 25298 heartbeater.cc:347] Connected to a master server at 127.24.83.62:41261
I20260806 12:56:33.140743 25298 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:33.140959 25298 heartbeater.cc:511] Master 127.24.83.62:41261 requested a full tablet report, sending...
I20260806 12:56:33.141353 24961 ts_manager.cc:194] Registered new tserver with Master: a7740acac67b46cba36b635049f143e9 (127.24.83.2:39881)
I20260806 12:56:33.141923 24961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:55931
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:56:33.163179 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41261
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:33.272746 25303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:33.272971 25303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:33.273041 25303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:33.274215 25303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:33.274252 25303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:33.274278 25303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:56:33.275969 25303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41261
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:33.276166 25303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:33.276353 25303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:33.281981 25309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:33.281986 25308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:33.282006 25311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:33.282207 25303 server_base.cc:1034] running on GCE node
I20260806 12:56:33.282461 25303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:33.282692 25303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:33.283964 25303 hybrid_clock.cc:648] HybridClock initialized: now 1786020993283948 us; error 24 us; skew 500 ppm
Aug 06 12:56:33 dist-test-slave-zkpd krb5kdc[24915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020993, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:33.286957 25303 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:56:33.287469 25303 webserver.cc:460] Webserver started at http://127.24.83.3:37505/ using document root <none> and password file <none>
I20260806 12:56:33.287717 25303 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:33.287786 25303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:33.287984 25303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:33.289049 25303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "d7e133d692804afd9988a2d611793862"
format_stamp: "Formatted at 2026-08-06 12:56:33 on dist-test-slave-zkpd"
server_key: "f3a95b6689c508a6b14e311ff5de4393"
server_key_iv: "2a9baaffbca5cfd6d91f237c299882ee"
server_key_version: "encryptionkey@0"
I20260806 12:56:33.289456 25303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "d7e133d692804afd9988a2d611793862"
format_stamp: "Formatted at 2026-08-06 12:56:33 on dist-test-slave-zkpd"
server_key: "f3a95b6689c508a6b14e311ff5de4393"
server_key_iv: "2a9baaffbca5cfd6d91f237c299882ee"
server_key_version: "encryptionkey@0"
I20260806 12:56:33.290912 25303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:33.291714 25318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:33.291921 25303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:33.292033 25303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "d7e133d692804afd9988a2d611793862"
format_stamp: "Formatted at 2026-08-06 12:56:33 on dist-test-slave-zkpd"
server_key: "f3a95b6689c508a6b14e311ff5de4393"
server_key_iv: "2a9baaffbca5cfd6d91f237c299882ee"
server_key_version: "encryptionkey@0"
I20260806 12:56:33.292115 25303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:33.301136 25303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:33.303427 25303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:33.303570 25303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:33.303732 25303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:33.304230 25303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:33.304296 25303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:33.304342 25303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:33.304369 25303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:33.317490 25303 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:41857
I20260806 12:56:33.317548 25434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:41857 every 8 connection(s)
I20260806 12:56:33.317906 25303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:56:33.318838 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 25303
I20260806 12:56:33.318923 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:56:33.319092 24908 external_mini_cluster.cc:1428] Setting key d983714ca3ef228c9b641b35dff469b9
Aug 06 12:56:33 dist-test-slave-zkpd krb5kdc[24915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020993, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:33.327702 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:33.318658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:60491 (local address 127.24.83.62:41261)
0806 12:56:33.318725 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:33.318727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:33.319118 (+ 391us) server_negotiation.cc:407] Connection header received
0806 12:56:33.320026 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:33.320028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:33.320040 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:33.320093 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:33.321196 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.321713 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.322697 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.322932 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.323017 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:33.325241 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:33.325243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:33.325243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:33.325250 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:33.326385 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:33.326772 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:33.326774 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:33.326774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:33.326833 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:33.327083 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:33.327084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:33.327084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:33.327197 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:33.327275 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:33.327534 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 12:56:33.327614 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 12:56:33.328013 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:33.318644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:41261 (local address 127.24.83.3:60491)
0806 12:56:33.318971 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:33.319024 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:33.319848 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:33.320163 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:33.320167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:33.320802 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:33.321098 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:33.321103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.321823 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.321824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:33.322606 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:33.322608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.323027 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.323028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:33.323158 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:33.323648 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:33.323656 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:33.325146 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:33.326474 (+ 1328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:33.326480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:33.326482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:33.326685 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:33.326914 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:33.326915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:33.326915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:33.327014 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:33.327264 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:33.327271 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:33.327408 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 12:56:33.327512 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 12:56:33.327657 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":216,"threads_started":1}
I20260806 12:56:33.328485 25435 heartbeater.cc:347] Connected to a master server at 127.24.83.62:41261
I20260806 12:56:33.328581 25435 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:33.328754 25435 heartbeater.cc:511] Master 127.24.83.62:41261 requested a full tablet report, sending...
I20260806 12:56:33.329120 24961 ts_manager.cc:194] Registered new tserver with Master: d7e133d692804afd9988a2d611793862 (127.24.83.3:41857)
I20260806 12:56:33.329496 24961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:60491
I20260806 12:56:33.331053 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 12:56:33 dist-test-slave-zkpd krb5kdc[24915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 12:56:33 dist-test-slave-zkpd krb5kdc[24915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:56:33.354863 25442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:33.346751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59618 (local address 127.24.83.1:44637)
0806 12:56:33.347038 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:33.347042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:33.347053 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:56:33.347077 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:33.347079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:33.347093 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:33.347190 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:33.347836 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.348740 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.349483 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.349798 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.349850 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:33.352221 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:33.352228 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:33.352228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:33.352235 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:33.353827 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:33.354125 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:33.354128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:33.354128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:33.354262 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:33.354416 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:33.354416 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:33.354416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:33.354528 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:33.354621 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:33.354685 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 12:56:33.354743 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":139,"threads_started":1}
W20260806 12:56:33.355376 25069 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59618
Aug 06 12:56:33 dist-test-slave-zkpd krb5kdc[24915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 12:56:33 dist-test-slave-zkpd krb5kdc[24915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:56:33.379626 25442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:33.372484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59634 (local address 127.24.83.1:44637)
0806 12:56:33.372558 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:33.372559 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:33.372606 (+ 47us) server_negotiation.cc:407] Connection header received
0806 12:56:33.372651 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:33.372652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:33.372661 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:33.372709 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:33.373351 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.373825 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.374512 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:33.374699 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:33.374769 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:33.377113 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:33.377115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:33.377115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:33.377123 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:33.378382 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:33.378790 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:33.378791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:33.378792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:33.378847 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:33.379045 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:33.379046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:33.379046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:33.379145 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:33.379278 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:33.379371 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 12:56:33.379496 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:56:33.380167 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 25023
I20260806 12:56:33.386375 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 25165
I20260806 12:56:33.392081 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 25303
I20260806 12:56:33.396763 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 24928
2026-08-06T12:56:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2061 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:56:33 dist-test-slave-zkpd krb5kdc[25449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:56:33 dist-test-slave-zkpd krb5kdc[25449](info): set up 2 sockets
Aug 06 12:56:33 dist-test-slave-zkpd krb5kdc[25449](info): commencing operation
krb5kdc: starting...
W20260806 12:56:34.441938 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:56:34 dist-test-slave-zkpd krb5kdc[25449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:56:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:56:34.554171 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:38669
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:38669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:34.650533 25462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:34.650755 25462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:34.650794 25462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:34.652002 25462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:34.652061 25462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:34.652086 25462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:34.652113 25462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:34.652141 25462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:34.653566 25462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:38669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:38669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:34.653861 25462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:34.654098 25462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:34.658833 25467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:34.658818 25468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:34.658803 25470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:34.658854 25462 server_base.cc:1034] running on GCE node
I20260806 12:56:34.659298 25462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:34.659633 25462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:34.660806 25462 hybrid_clock.cc:648] HybridClock initialized: now 1786020994660791 us; error 28 us; skew 500 ppm
Aug 06 12:56:34 dist-test-slave-zkpd krb5kdc[25449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020994, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:34.663842 25462 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:56:34.664305 25462 webserver.cc:460] Webserver started at http://127.24.83.62:32797/ using document root <none> and password file <none>
I20260806 12:56:34.664521 25462 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:34.664587 25462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:34.664731 25462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:34.665702 25462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "2570e843604e482b923e50e356102f0c"
format_stamp: "Formatted at 2026-08-06 12:56:34 on dist-test-slave-zkpd"
server_key: "fa31934815fc15261d6b184e4750a585"
server_key_iv: "3cd363c56fc7eac2d872f8b090852a03"
server_key_version: "encryptionkey@0"
I20260806 12:56:34.666054 25462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "2570e843604e482b923e50e356102f0c"
format_stamp: "Formatted at 2026-08-06 12:56:34 on dist-test-slave-zkpd"
server_key: "fa31934815fc15261d6b184e4750a585"
server_key_iv: "3cd363c56fc7eac2d872f8b090852a03"
server_key_version: "encryptionkey@0"
I20260806 12:56:34.667385 25462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 12:56:34.668119 25477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:34.668303 25462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:34.668414 25462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "2570e843604e482b923e50e356102f0c"
format_stamp: "Formatted at 2026-08-06 12:56:34 on dist-test-slave-zkpd"
server_key: "fa31934815fc15261d6b184e4750a585"
server_key_iv: "3cd363c56fc7eac2d872f8b090852a03"
server_key_version: "encryptionkey@0"
I20260806 12:56:34.668516 25462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:34.683543 25462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:34.685314 25462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:34.685474 25462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:34.689625 25462 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:38669
I20260806 12:56:34.689666 25531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:38669 every 8 connection(s)
I20260806 12:56:34.690025 25462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:34.691107 25532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:34.693437 25532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c: Bootstrap starting.
I20260806 12:56:34.694020 25532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:34.694319 25532 log.cc:824] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:34.695026 25532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c: No bootstrap required, opened a new log
I20260806 12:56:34.696581 25532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2570e843604e482b923e50e356102f0c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38669 } }
I20260806 12:56:34.696708 25532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:34.696766 25532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2570e843604e482b923e50e356102f0c, State: Initialized, Role: FOLLOWER
I20260806 12:56:34.696916 25532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2570e843604e482b923e50e356102f0c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38669 } }
I20260806 12:56:34.696977 25532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:34.697018 25532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:34.697067 25532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:34.697541 25532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2570e843604e482b923e50e356102f0c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38669 } }
I20260806 12:56:34.697654 25532 leader_election.cc:302] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2570e843604e482b923e50e356102f0c; no voters:
I20260806 12:56:34.697868 25532 leader_election.cc:288] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:34.697921 25535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:34.698096 25535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [term 1 LEADER]: Becoming Leader. State: Replica: 2570e843604e482b923e50e356102f0c, State: Running, Role: LEADER
I20260806 12:56:34.698158 25532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:34.698241 25535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2570e843604e482b923e50e356102f0c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38669 } }
I20260806 12:56:34.698423 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 25462
I20260806 12:56:34.698490 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:34.698678 24908 external_mini_cluster.cc:1428] Setting key d01bb9623fd63f0c374132646d7a8faf
I20260806 12:56:34.698673 25536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2570e843604e482b923e50e356102f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2570e843604e482b923e50e356102f0c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38669 } } }
I20260806 12:56:34.698747 25536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:34.698894 25537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2570e843604e482b923e50e356102f0c. Latest consensus state: current_term: 1 leader_uuid: "2570e843604e482b923e50e356102f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2570e843604e482b923e50e356102f0c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38669 } } }
I20260806 12:56:34.698961 25539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:34.698972 25537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:34.699702 25539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:34.701294 25539 catalog_manager.cc:1347] Generated new cluster ID: e0d2cb3c0597458fa839bf523144d2a7
I20260806 12:56:34.701344 25539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:56:34 dist-test-slave-zkpd krb5kdc[25449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:34.711051 25539 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:34.711238 25543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:34.699195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39858 (local address 127.24.83.62:38669)
0806 12:56:34.699467 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:34.699476 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:34.699511 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:56:34.699637 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:34.699645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:34.699712 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:34.700574 (+ 862us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:34.700672 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:34.702124 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:34.703283 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:34.703876 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:34.704001 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:34.706735 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:34.706739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:34.706740 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:34.706750 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:34.709554 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:34.709945 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:34.709947 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:34.709948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:34.710058 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:34.710270 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:34.710270 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:34.710271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:34.710521 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:34.710696 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:34.710790 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 12:56:34.710936 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":95,"threads_started":1}
I20260806 12:56:34.711787 25539 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:34.727843 25539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2570e843604e482b923e50e356102f0c: Generated new TSK 0
I20260806 12:56:34.728055 25539 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:56:34.755566 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:38669
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:34.838617 25560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:34.838850 25560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:34.838888 25560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:34.840058 25560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:34.840121 25560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:34.840184 25560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:56:34.841665 25560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:38669
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:34.841928 25560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:34.842150 25560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:34.847026 25566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:34.847040 25568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:34.847085 25565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:34.847149 25560 server_base.cc:1034] running on GCE node
I20260806 12:56:34.847440 25560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:34.847679 25560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:34.848839 25560 hybrid_clock.cc:648] HybridClock initialized: now 1786020994848817 us; error 60 us; skew 500 ppm
Aug 06 12:56:34 dist-test-slave-zkpd krb5kdc[25449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020994, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:34.851142 25560 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:56:34.851603 25560 webserver.cc:460] Webserver started at http://127.24.83.1:35761/ using document root <none> and password file <none>
I20260806 12:56:34.851838 25560 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:34.851927 25560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:34.852084 25560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:34.853070 25560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "2e31d7003fe549cc8a98590cde0b4dae"
format_stamp: "Formatted at 2026-08-06 12:56:34 on dist-test-slave-zkpd"
server_key: "9aa9385528f2116eb757e6df53a62a53"
server_key_iv: "087fb91b5f6ed55b63ba054ba77ff3bb"
server_key_version: "encryptionkey@0"
I20260806 12:56:34.853430 25560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "2e31d7003fe549cc8a98590cde0b4dae"
format_stamp: "Formatted at 2026-08-06 12:56:34 on dist-test-slave-zkpd"
server_key: "9aa9385528f2116eb757e6df53a62a53"
server_key_iv: "087fb91b5f6ed55b63ba054ba77ff3bb"
server_key_version: "encryptionkey@0"
I20260806 12:56:34.854815 25560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:56:34.855602 25575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:34.855775 25560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:56:34.855871 25560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "2e31d7003fe549cc8a98590cde0b4dae"
format_stamp: "Formatted at 2026-08-06 12:56:34 on dist-test-slave-zkpd"
server_key: "9aa9385528f2116eb757e6df53a62a53"
server_key_iv: "087fb91b5f6ed55b63ba054ba77ff3bb"
server_key_version: "encryptionkey@0"
I20260806 12:56:34.856000 25560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:34.865819 25560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:34.867491 25560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:34.867640 25560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:34.867846 25560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:34.868161 25560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:34.868214 25560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:34.868255 25560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:34.868283 25560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:34.874325 25560 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:33685
I20260806 12:56:34.874379 25690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:33685 every 8 connection(s)
I20260806 12:56:34.874652 25560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
Aug 06 12:56:34 dist-test-slave-zkpd krb5kdc[25449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020994, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:34.880455 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 25560
I20260806 12:56:34.880538 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:56:34.880692 24908 external_mini_cluster.cc:1428] Setting key b083127f02d83b449d7dccf5798c0079
I20260806 12:56:34.883850 25543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:34.875326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:48199 (local address 127.24.83.62:38669)
0806 12:56:34.875371 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:34.875373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:34.875600 (+ 227us) server_negotiation.cc:407] Connection header received
0806 12:56:34.876580 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:34.876581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:34.876601 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:34.876654 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:34.877207 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:34.877706 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:34.878637 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:34.878854 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:34.878903 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:34.880843 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:34.880845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:34.880846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:34.880854 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:34.882467 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:34.882882 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:34.882890 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:34.882891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:34.882949 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:34.883198 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:34.883199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:34.883200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:34.883312 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:34.883385 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:34.883670 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 12:56:34.883761 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 12:56:34.884012 25693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:34.875314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:38669 (local address 127.24.83.1:48199)
0806 12:56:34.875475 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:34.875508 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:34.876493 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:34.876726 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:34.876729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:34.876958 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:34.877136 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:34.877140 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:34.877780 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:34.877780 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:34.878555 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:34.878556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:34.878926 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:34.878927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:34.879020 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:34.879435 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:34.879443 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:34.880742 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:34.882551 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:34.882557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:34.882559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:34.882792 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:34.883023 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:34.883024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:34.883025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:34.883132 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:34.883394 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:34.883401 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:34.883533 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 12:56:34.883630 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 12:56:34.883779 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260806 12:56:34.884485 25691 heartbeater.cc:347] Connected to a master server at 127.24.83.62:38669
I20260806 12:56:34.884595 25691 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:34.884797 25691 heartbeater.cc:511] Master 127.24.83.62:38669 requested a full tablet report, sending...
I20260806 12:56:34.885267 25495 ts_manager.cc:194] Registered new tserver with Master: 2e31d7003fe549cc8a98590cde0b4dae (127.24.83.1:33685)
I20260806 12:56:34.885850 25495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:48199
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:56:34.906343 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:38669
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:34.987353 25696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:34.987542 25696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:34.987565 25696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:34.988773 25696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:34.988817 25696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:34.988843 25696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:56:34.990363 25696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:38669
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:34.990566 25696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:34.990792 25696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:34.995649 25702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:34.995616 25701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:34.995627 25704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:34.995690 25696 server_base.cc:1034] running on GCE node
I20260806 12:56:34.996063 25696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:34.996313 25696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:34.997473 25696 hybrid_clock.cc:648] HybridClock initialized: now 1786020994997467 us; error 34 us; skew 500 ppm
Aug 06 12:56:34 dist-test-slave-zkpd krb5kdc[25449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020994, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:35.000022 25696 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:56:35.000434 25696 webserver.cc:460] Webserver started at http://127.24.83.2:43711/ using document root <none> and password file <none>
I20260806 12:56:35.000607 25696 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:35.000646 25696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:35.000732 25696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:35.001677 25696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "93ef5792891841409b06a92711819ebd"
format_stamp: "Formatted at 2026-08-06 12:56:34 on dist-test-slave-zkpd"
server_key: "7ebf95b2dd70a12a166bfdb0b1e13359"
server_key_iv: "456ff100d7bc107f1d61fa739b15dc5e"
server_key_version: "encryptionkey@0"
I20260806 12:56:35.001973 25696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "93ef5792891841409b06a92711819ebd"
format_stamp: "Formatted at 2026-08-06 12:56:34 on dist-test-slave-zkpd"
server_key: "7ebf95b2dd70a12a166bfdb0b1e13359"
server_key_iv: "456ff100d7bc107f1d61fa739b15dc5e"
server_key_version: "encryptionkey@0"
I20260806 12:56:35.003206 25696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:35.004004 25711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:35.004154 25696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:35.004244 25696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "93ef5792891841409b06a92711819ebd"
format_stamp: "Formatted at 2026-08-06 12:56:34 on dist-test-slave-zkpd"
server_key: "7ebf95b2dd70a12a166bfdb0b1e13359"
server_key_iv: "456ff100d7bc107f1d61fa739b15dc5e"
server_key_version: "encryptionkey@0"
I20260806 12:56:35.004304 25696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:35.011788 25696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:35.013499 25696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:35.013605 25696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:35.013788 25696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:35.014017 25696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:35.014046 25696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:35.014066 25696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:35.014079 25696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:35.019538 25696 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:39565
I20260806 12:56:35.019603 25831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:39565 every 8 connection(s)
I20260806 12:56:35.019954 25696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:56:35.020704 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 25696
I20260806 12:56:35.020790 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:56:35.020944 24908 external_mini_cluster.cc:1428] Setting key 5495bf98f75a8b003c41d79a9bcb1973
Aug 06 12:56:35 dist-test-slave-zkpd krb5kdc[25449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020994, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
I20260806 12:56:35.029919 25543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.020747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:50041 (local address 127.24.83.62:38669)
0806 12:56:35.020829 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:35.020831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:35.021039 (+ 208us) server_negotiation.cc:407] Connection header received
0806 12:56:35.022080 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:35.022082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:35.022095 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:35.022158 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:35.022837 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.023333 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.024412 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.024607 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.024693 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.027049 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:35.027051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:35.027052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:35.027059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:35.028550 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.028966 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.028967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.028968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.029028 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.029350 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.029351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.029351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.029455 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:35.029507 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:35.029760 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 12:56:35.029829 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 12:56:35.030192 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.020667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:38669 (local address 127.24.83.2:50041)
0806 12:56:35.020894 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:35.020932 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:35.021966 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:35.022225 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:35.022228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:35.022528 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:35.022708 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.022712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.023428 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.023429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.024311 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.024314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.024705 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.024706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.024834 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.025341 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:35.025349 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:35.026946 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:35.028644 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:35.028650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:35.028652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:35.028874 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:35.029130 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:35.029132 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:35.029132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:35.029273 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:35.029532 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:35.029537 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:35.029650 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 12:56:35.029734 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 12:56:35.029847 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
I20260806 12:56:35.030700 25832 heartbeater.cc:347] Connected to a master server at 127.24.83.62:38669
I20260806 12:56:35.030773 25832 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:35.030959 25832 heartbeater.cc:511] Master 127.24.83.62:38669 requested a full tablet report, sending...
I20260806 12:56:35.031318 25495 ts_manager.cc:194] Registered new tserver with Master: 93ef5792891841409b06a92711819ebd (127.24.83.2:39565)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
I20260806 12:56:35.031803 25495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:50041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:56:35.046330 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:38669
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:35.129263 25837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:35.129453 25837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:35.129475 25837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:35.130858 25837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:35.130915 25837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:35.130944 25837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:56:35.132652 25837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:38669
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:35.132915 25837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:35.133169 25837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:35.138059 25846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:35.138087 25843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:35.138059 25842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:35.138213 25837 server_base.cc:1034] running on GCE node
I20260806 12:56:35.138538 25837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:35.138734 25837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:35.139941 25837 hybrid_clock.cc:648] HybridClock initialized: now 1786020995139922 us; error 35 us; skew 500 ppm
Aug 06 12:56:35 dist-test-slave-zkpd krb5kdc[25449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020995, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:35.142347 25837 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:56:35.142748 25837 webserver.cc:460] Webserver started at http://127.24.83.3:39773/ using document root <none> and password file <none>
I20260806 12:56:35.142982 25837 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:35.143049 25837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:35.143198 25837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:35.144367 25837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "f611ea5f1ad849a49acf2cf58021efd6"
format_stamp: "Formatted at 2026-08-06 12:56:35 on dist-test-slave-zkpd"
server_key: "52d5ad4ac053e9f98d5e380c8fab975d"
server_key_iv: "7e0d55cba876836f700b47c50440579b"
server_key_version: "encryptionkey@0"
I20260806 12:56:35.144724 25837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "f611ea5f1ad849a49acf2cf58021efd6"
format_stamp: "Formatted at 2026-08-06 12:56:35 on dist-test-slave-zkpd"
server_key: "52d5ad4ac053e9f98d5e380c8fab975d"
server_key_iv: "7e0d55cba876836f700b47c50440579b"
server_key_version: "encryptionkey@0"
I20260806 12:56:35.146029 25837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:56:35.146711 25854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:35.146879 25837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:35.146982 25837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "f611ea5f1ad849a49acf2cf58021efd6"
format_stamp: "Formatted at 2026-08-06 12:56:35 on dist-test-slave-zkpd"
server_key: "52d5ad4ac053e9f98d5e380c8fab975d"
server_key_iv: "7e0d55cba876836f700b47c50440579b"
server_key_version: "encryptionkey@0"
I20260806 12:56:35.147065 25837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:35.158250 25837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:35.160184 25837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:35.160339 25837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:35.160558 25837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:35.160866 25837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:35.160931 25837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:35.160974 25837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:35.161001 25837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:35.166728 25837 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:40489
I20260806 12:56:35.166775 25970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:40489 every 8 connection(s)
I20260806 12:56:35.167084 25837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:56:35.170939 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 25837
I20260806 12:56:35.171037 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:56:35.171202 24908 external_mini_cluster.cc:1428] Setting key 78ff8760ea79c3d3a7741226a581bd77
Aug 06 12:56:35 dist-test-slave-zkpd krb5kdc[25449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020995, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:35.176501 25543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.167727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:60473 (local address 127.24.83.62:38669)
0806 12:56:35.167776 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:35.167778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:35.168042 (+ 264us) server_negotiation.cc:407] Connection header received
0806 12:56:35.169050 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:35.169051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:35.169062 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:35.169114 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:35.169664 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.170142 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.171480 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.171830 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.171926 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.173919 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:35.173921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:35.173921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:35.173927 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:35.175347 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.175681 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.175682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.175682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.175732 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.175977 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.175978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.175978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.176077 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:35.176130 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:35.176372 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 12:56:35.176440 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 12:56:35.176648 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.167717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:38669 (local address 127.24.83.3:60473)
0806 12:56:35.167866 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:35.167954 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:35.168936 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:35.169169 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:35.169172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:35.169411 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:35.169594 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.169598 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.170219 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.170220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.171295 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.171298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.171942 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.171944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.172093 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.172525 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:35.172532 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:35.173834 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:35.175419 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:35.175423 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:35.175425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:35.175619 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:35.175790 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:35.175791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:35.175791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:35.175882 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:35.176136 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:35.176140 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:35.176247 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 12:56:35.176329 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 12:56:35.176443 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":97,"threads_started":1}
I20260806 12:56:35.177132 25971 heartbeater.cc:347] Connected to a master server at 127.24.83.62:38669
I20260806 12:56:35.177211 25971 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:35.177346 25971 heartbeater.cc:511] Master 127.24.83.62:38669 requested a full tablet report, sending...
I20260806 12:56:35.177675 25495 ts_manager.cc:194] Registered new tserver with Master: f611ea5f1ad849a49acf2cf58021efd6 (127.24.83.3:40489)
I20260806 12:56:35.178013 25495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:60473
I20260806 12:56:35.183007 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:56:35.190207 25543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.184918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.24.83.62:38669)
0806 12:56:35.184992 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:35.184993 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:35.185017 (+ 24us) server_negotiation.cc:407] Connection header received
0806 12:56:35.185068 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:35.185068 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:35.185076 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:35.185120 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:35.185600 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.186031 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.186684 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.186861 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.186913 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.187874 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:35.187875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:35.187875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:35.187881 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:35.189276 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.189570 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.189570 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.189571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.189618 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.189769 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.189769 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.189769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.189897 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:35.189969 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:35.190051 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 12:56:35.190105 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 12:56:35.192953 25495 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:56:35.194783 25495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:56:35.201753 25983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.197527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:33685 (local address 127.0.0.1:32930)
0806 12:56:35.197836 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:35.197844 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:35.197908 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:35.198328 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:35.198329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:35.198331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:35.198390 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.198392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.199361 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.199362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.200618 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.200621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.201436 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.201437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.201518 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.201535 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 12:56:35.201618 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 12:56:35.201672 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":85,"threads_started":1}
I20260806 12:56:35.201761 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.197686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32930 (local address 127.24.83.1:33685)
0806 12:56:35.198160 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:35.198163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:35.198172 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:56:35.198203 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:35.198205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:35.198216 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:35.198325 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:35.198446 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.199261 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.200750 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.201345 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.201435 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.201495 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:35.201646 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 12:56:35.201686 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":81,"threads_started":1}
I20260806 12:56:35.202039 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.197583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40432 (local address 127.24.83.3:40489)
0806 12:56:35.197751 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:35.197756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:35.197873 (+ 117us) server_negotiation.cc:407] Connection header received
0806 12:56:35.198009 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:35.198011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:35.198027 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:35.198129 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:35.198418 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.199709 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.200904 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.201577 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.201648 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.201699 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:35.201917 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 12:56:35.201960 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260806 12:56:35.202030 25979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.197754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:39565 (local address 127.0.0.1:58964)
0806 12:56:35.197880 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:35.197891 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:35.197945 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:35.198415 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:35.198416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:35.198418 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:35.198451 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.198452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.199800 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.199802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.200924 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.200927 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.201818 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.201819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.201878 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.201883 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:56:35.201928 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 12:56:35.201963 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":2,"thread_start_us":51,"threads_started":1}
I20260806 12:56:35.202239 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.197498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:40489 (local address 127.0.0.1:40432)
0806 12:56:35.197768 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:35.197807 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:35.197908 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:35.198247 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:35.198250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:35.198253 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:35.198320 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.198322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.199817 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.199817 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.200800 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.200803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.201747 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.201748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.201812 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.201816 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:56:35.202116 (+ 300us) client_negotiation.cc:240] Negotiation successful
0806 12:56:35.202162 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260806 12:56:35.202462 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.197959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58964 (local address 127.24.83.2:39565)
0806 12:56:35.198162 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:35.198166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:35.198176 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:56:35.198216 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:35.198218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:35.198234 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:35.198307 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:35.198541 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.199684 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.201004 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.201681 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.201741 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.201789 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:35.202236 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 12:56:35.202285 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":86,"threads_started":1}
I20260806 12:56:35.202692 25623 tablet_service.cc:1467] Processing CreateTablet for tablet d8e1efef0f1f4865bc25af8a1d4532b7 (DEFAULT_TABLE table=test-table [id=4bf7cc9629b24804a6327deb1aec1c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:35.203097 25623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e1efef0f1f4865bc25af8a1d4532b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:35.203145 25762 tablet_service.cc:1467] Processing CreateTablet for tablet d8e1efef0f1f4865bc25af8a1d4532b7 (DEFAULT_TABLE table=test-table [id=4bf7cc9629b24804a6327deb1aec1c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:35.203433 25762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e1efef0f1f4865bc25af8a1d4532b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:35.205928 25985 tablet_bootstrap.cc:492] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae: Bootstrap starting.
I20260806 12:56:35.206143 25986 tablet_bootstrap.cc:492] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd: Bootstrap starting.
I20260806 12:56:35.206317 25903 tablet_service.cc:1467] Processing CreateTablet for tablet d8e1efef0f1f4865bc25af8a1d4532b7 (DEFAULT_TABLE table=test-table [id=4bf7cc9629b24804a6327deb1aec1c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:35.206657 25903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e1efef0f1f4865bc25af8a1d4532b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:35.206769 25986 tablet_bootstrap.cc:654] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:35.206779 25985 tablet_bootstrap.cc:654] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:35.207602 25985 log.cc:824] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:35.208410 25985 tablet_bootstrap.cc:492] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae: No bootstrap required, opened a new log
I20260806 12:56:35.208485 25985 ts_tablet_manager.cc:1397] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 12:56:35.209158 25986 log.cc:824] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:35.210338 25989 tablet_bootstrap.cc:492] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6: Bootstrap starting.
I20260806 12:56:35.210906 25989 tablet_bootstrap.cc:654] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:35.211030 25985 raft_consensus.cc:348] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f611ea5f1ad849a49acf2cf58021efd6" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40489 } } peers { permanent_uuid: "2e31d7003fe549cc8a98590cde0b4dae" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33685 } } peers { permanent_uuid: "93ef5792891841409b06a92711819ebd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39565 } }
I20260806 12:56:35.211242 25985 raft_consensus.cc:374] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:35.211318 25985 raft_consensus.cc:729] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e31d7003fe549cc8a98590cde0b4dae, State: Initialized, Role: FOLLOWER
I20260806 12:56:35.211463 25985 consensus_queue.cc:260] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f611ea5f1ad849a49acf2cf58021efd6" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40489 } } peers { permanent_uuid: "2e31d7003fe549cc8a98590cde0b4dae" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33685 } } peers { permanent_uuid: "93ef5792891841409b06a92711819ebd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39565 } }
I20260806 12:56:35.211759 25989 log.cc:824] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:35.211966 25691 heartbeater.cc:503] Master 127.24.83.62:38669 was elected leader, sending a full tablet report...
I20260806 12:56:35.211974 25985 ts_tablet_manager.cc:1428] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 12:56:35.212544 25986 tablet_bootstrap.cc:492] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd: No bootstrap required, opened a new log
I20260806 12:56:35.212630 25989 tablet_bootstrap.cc:492] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6: No bootstrap required, opened a new log
I20260806 12:56:35.212639 25986 ts_tablet_manager.cc:1397] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 12:56:35.212677 25989 ts_tablet_manager.cc:1397] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 12:56:35.215265 25989 raft_consensus.cc:348] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f611ea5f1ad849a49acf2cf58021efd6" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40489 } } peers { permanent_uuid: "2e31d7003fe549cc8a98590cde0b4dae" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33685 } } peers { permanent_uuid: "93ef5792891841409b06a92711819ebd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39565 } }
I20260806 12:56:35.215240 25986 raft_consensus.cc:348] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f611ea5f1ad849a49acf2cf58021efd6" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40489 } } peers { permanent_uuid: "2e31d7003fe549cc8a98590cde0b4dae" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33685 } } peers { permanent_uuid: "93ef5792891841409b06a92711819ebd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39565 } }
I20260806 12:56:35.215391 25989 raft_consensus.cc:374] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:35.215399 25986 raft_consensus.cc:374] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:35.215413 25989 raft_consensus.cc:729] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f611ea5f1ad849a49acf2cf58021efd6, State: Initialized, Role: FOLLOWER
I20260806 12:56:35.215422 25986 raft_consensus.cc:729] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ef5792891841409b06a92711819ebd, State: Initialized, Role: FOLLOWER
I20260806 12:56:35.215518 25989 consensus_queue.cc:260] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f611ea5f1ad849a49acf2cf58021efd6" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40489 } } peers { permanent_uuid: "2e31d7003fe549cc8a98590cde0b4dae" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33685 } } peers { permanent_uuid: "93ef5792891841409b06a92711819ebd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39565 } }
I20260806 12:56:35.215533 25986 consensus_queue.cc:260] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f611ea5f1ad849a49acf2cf58021efd6" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40489 } } peers { permanent_uuid: "2e31d7003fe549cc8a98590cde0b4dae" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33685 } } peers { permanent_uuid: "93ef5792891841409b06a92711819ebd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39565 } }
I20260806 12:56:35.215871 25832 heartbeater.cc:503] Master 127.24.83.62:38669 was elected leader, sending a full tablet report...
I20260806 12:56:35.215987 25971 heartbeater.cc:503] Master 127.24.83.62:38669 was elected leader, sending a full tablet report...
I20260806 12:56:35.215984 25986 ts_tablet_manager.cc:1428] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 12:56:35.215802 25989 ts_tablet_manager.cc:1428] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260806 12:56:35.270493 25833 tablet.cc:2369] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:56:35.355387 25993 raft_consensus.cc:482] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:56:35.355502 25993 raft_consensus.cc:504] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f611ea5f1ad849a49acf2cf58021efd6" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40489 } } peers { permanent_uuid: "2e31d7003fe549cc8a98590cde0b4dae" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33685 } } peers { permanent_uuid: "93ef5792891841409b06a92711819ebd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39565 } }
I20260806 12:56:35.356172 25993 leader_election.cc:288] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f611ea5f1ad849a49acf2cf58021efd6 (127.24.83.3:40489), 2e31d7003fe549cc8a98590cde0b4dae (127.24.83.1:33685)
I20260806 12:56:35.360303 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.356537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:46131 (local address 127.24.83.3:40489)
0806 12:56:35.356601 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:35.356604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:35.356610 (+ 6us) server_negotiation.cc:407] Connection header received
0806 12:56:35.356632 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:35.356633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:35.356639 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:35.356679 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:35.356918 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.357633 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.359217 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.359843 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.359954 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.359967 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:35.360190 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 12:56:35.360232 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 12:56:35.360303 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.356358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:40489 (local address 127.24.83.2:46131)
0806 12:56:35.356410 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:35.356424 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:35.356473 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:35.356713 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:35.356715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:35.356718 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:35.356837 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.356840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.357708 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.357708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.359113 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.359116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.359980 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.359981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.360119 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.360125 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 12:56:35.360183 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 12:56:35.360224 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":50}
I20260806 12:56:35.360378 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.356449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:37835 (local address 127.24.83.1:33685)
0806 12:56:35.356497 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:35.356500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:35.356739 (+ 239us) server_negotiation.cc:407] Connection header received
0806 12:56:35.356776 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:35.356776 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:35.356787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:35.356827 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:35.357097 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.357786 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.359347 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.359972 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.360033 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.360042 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:35.360272 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 12:56:35.360306 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 12:56:35.360421 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.356394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:33685 (local address 127.24.83.2:37835)
0806 12:56:35.356635 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:35.356644 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:35.356721 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:35.356851 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:35.356851 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:35.356853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:35.357020 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.357022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.357850 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.357851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.359265 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:35.359267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.360039 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.360040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:35.360148 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.360152 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:56:35.360258 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 12:56:35.360301 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":101,"threads_started":1}
I20260806 12:56:35.360795 25925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e1efef0f1f4865bc25af8a1d4532b7" candidate_uuid: "93ef5792891841409b06a92711819ebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f611ea5f1ad849a49acf2cf58021efd6" is_pre_election: true
I20260806 12:56:35.360960 25644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e1efef0f1f4865bc25af8a1d4532b7" candidate_uuid: "93ef5792891841409b06a92711819ebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e31d7003fe549cc8a98590cde0b4dae" is_pre_election: true
I20260806 12:56:35.361053 25925 raft_consensus.cc:2454] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93ef5792891841409b06a92711819ebd in term 0.
I20260806 12:56:35.361159 25644 raft_consensus.cc:2454] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93ef5792891841409b06a92711819ebd in term 0.
I20260806 12:56:35.361488 25714 leader_election.cc:302] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e31d7003fe549cc8a98590cde0b4dae, 93ef5792891841409b06a92711819ebd; no voters:
I20260806 12:56:35.361672 25993 raft_consensus.cc:2781] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:56:35.361734 25993 raft_consensus.cc:482] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:56:35.361752 25993 raft_consensus.cc:3037] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:35.362480 25993 raft_consensus.cc:504] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f611ea5f1ad849a49acf2cf58021efd6" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40489 } } peers { permanent_uuid: "2e31d7003fe549cc8a98590cde0b4dae" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33685 } } peers { permanent_uuid: "93ef5792891841409b06a92711819ebd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39565 } }
I20260806 12:56:35.362625 25993 leader_election.cc:288] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [CANDIDATE]: Term 1 election: Requested vote from peers f611ea5f1ad849a49acf2cf58021efd6 (127.24.83.3:40489), 2e31d7003fe549cc8a98590cde0b4dae (127.24.83.1:33685)
I20260806 12:56:35.362802 25644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e1efef0f1f4865bc25af8a1d4532b7" candidate_uuid: "93ef5792891841409b06a92711819ebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e31d7003fe549cc8a98590cde0b4dae"
I20260806 12:56:35.362854 25925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e1efef0f1f4865bc25af8a1d4532b7" candidate_uuid: "93ef5792891841409b06a92711819ebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f611ea5f1ad849a49acf2cf58021efd6"
I20260806 12:56:35.362918 25925 raft_consensus.cc:3037] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:35.362910 25644 raft_consensus.cc:3037] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:35.364498 25644 raft_consensus.cc:2454] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93ef5792891841409b06a92711819ebd in term 1.
I20260806 12:56:35.364641 25925 raft_consensus.cc:2454] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93ef5792891841409b06a92711819ebd in term 1.
I20260806 12:56:35.364707 25714 leader_election.cc:302] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e31d7003fe549cc8a98590cde0b4dae, 93ef5792891841409b06a92711819ebd; no voters:
I20260806 12:56:35.364794 25993 raft_consensus.cc:2781] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:35.364948 25993 raft_consensus.cc:686] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [term 1 LEADER]: Becoming Leader. State: Replica: 93ef5792891841409b06a92711819ebd, State: Running, Role: LEADER
I20260806 12:56:35.365069 25993 consensus_queue.cc:237] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f611ea5f1ad849a49acf2cf58021efd6" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40489 } } peers { permanent_uuid: "2e31d7003fe549cc8a98590cde0b4dae" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33685 } } peers { permanent_uuid: "93ef5792891841409b06a92711819ebd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39565 } }
I20260806 12:56:35.366015 25495 catalog_manager.cc:5495] T d8e1efef0f1f4865bc25af8a1d4532b7 P 93ef5792891841409b06a92711819ebd reported cstate change: term changed from 0 to 1, leader changed from <none> to 93ef5792891841409b06a92711819ebd (127.24.83.2). New cstate: current_term: 1 leader_uuid: "93ef5792891841409b06a92711819ebd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f611ea5f1ad849a49acf2cf58021efd6" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e31d7003fe549cc8a98590cde0b4dae" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93ef5792891841409b06a92711819ebd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39565 } health_report { overall_health: HEALTHY } } }
W20260806 12:56:35.375308 25692 tablet.cc:2369] T d8e1efef0f1f4865bc25af8a1d4532b7 P 2e31d7003fe549cc8a98590cde0b4dae: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 12:56:35 dist-test-slave-zkpd krb5kdc[25449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:56:35.386087 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.377945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32942 (local address 127.24.83.1:33685)
0806 12:56:35.378011 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:35.378013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:35.378232 (+ 219us) server_negotiation.cc:407] Connection header received
0806 12:56:35.378262 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:35.378262 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:35.378271 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:35.378329 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:35.378916 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.379381 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.380136 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.380319 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.380370 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.382807 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:35.382815 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:35.382816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:35.382822 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:35.385125 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.385427 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.385430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.385430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.385482 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.385634 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.385634 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.385635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.385755 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:35.385863 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:35.385923 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 12:56:35.385983 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 12:56:35 dist-test-slave-zkpd krb5kdc[25449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020995, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 12:56:35 dist-test-slave-zkpd krb5kdc[25449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020995, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:56:35.409541 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.402141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32948 (local address 127.24.83.1:33685)
0806 12:56:35.402204 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:35.402207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:35.402317 (+ 110us) server_negotiation.cc:407] Connection header received
0806 12:56:35.402359 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:35.402360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:35.402368 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:35.402416 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:35.402917 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.403377 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.404115 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.404373 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.404436 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.406654 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:35.406655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:35.406656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:35.406663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:35.408242 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.408631 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.408633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.408633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.408690 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.408909 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.408910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.408910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.409053 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:35.409157 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:35.409361 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 12:56:35.409435 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 12:56:35.417672 25972 tablet.cc:2369] T d8e1efef0f1f4865bc25af8a1d4532b7 P f611ea5f1ad849a49acf2cf58021efd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 12:56:35 dist-test-slave-zkpd krb5kdc[25449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 12:56:35 dist-test-slave-zkpd krb5kdc[25449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:56:35.431252 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:35.424696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32956 (local address 127.24.83.1:33685)
0806 12:56:35.424757 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:35.424759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:35.424874 (+ 115us) server_negotiation.cc:407] Connection header received
0806 12:56:35.424925 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:35.424926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:35.424935 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:35.424983 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:35.425559 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.426003 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.426694 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:35.426859 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:35.426926 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:35.429032 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:35.429033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:35.429033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:35.429039 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:35.430425 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.430700 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.430700 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.430701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.430747 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:35.430885 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:35.430886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:35.430886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:35.430961 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:35.431028 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:35.431113 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 12:56:35.431164 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 12:56:35.432003 25603 tablet_service.cc:2908] Rejecting scan request for tablet d8e1efef0f1f4865bc25af8a1d4532b7: Uninitialized: clean time has not yet been initialized
I20260806 12:56:35.432996 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 25560
I20260806 12:56:35.438685 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 25696
I20260806 12:56:35.443609 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 25837
I20260806 12:56:35.448138 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 25462
2026-08-06T12:56:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2052 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:56:35 dist-test-slave-zkpd krb5kdc[26011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:56:35 dist-test-slave-zkpd krb5kdc[26011](info): set up 2 sockets
Aug 06 12:56:35 dist-test-slave-zkpd krb5kdc[26011](info): commencing operation
krb5kdc: starting...
W20260806 12:56:36.495311 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:56:36 dist-test-slave-zkpd krb5kdc[26011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:56:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:56:36.612175 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33351
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:33351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:36.692094 26024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:36.692281 26024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:36.692303 26024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:36.693372 26024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:36.693410 26024 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 12:56:36.693423 26024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:36.693436 26024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:36.693449 26024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:36.693460 26024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:36.694829 26024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:33351
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:36.695012 26024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:36.695171 26024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:36.699821 26033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:36.699813 26031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:36.700021 26024 server_base.cc:1034] running on GCE node
W20260806 12:56:36.700140 26030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:36.700361 26024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:36.700713 26024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:36.701880 26024 hybrid_clock.cc:648] HybridClock initialized: now 1786020996701849 us; error 42 us; skew 500 ppm
Aug 06 12:56:36 dist-test-slave-zkpd krb5kdc[26011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020996, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:36.704429 26024 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:56:36.704833 26024 webserver.cc:460] Webserver started at http://127.24.83.62:39345/ using document root <none> and password file <none>
I20260806 12:56:36.705046 26024 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:36.705111 26024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:36.705258 26024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:36.706346 26024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "5e7eb8cc13364a489a1945f4d44e508f"
format_stamp: "Formatted at 2026-08-06 12:56:36 on dist-test-slave-zkpd"
server_key: "fa16353e1eeb04359e752a1088197b07"
server_key_iv: "e6dc19d46dbbee0eba520125f8399200"
server_key_version: "encryptionkey@0"
I20260806 12:56:36.706694 26024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "5e7eb8cc13364a489a1945f4d44e508f"
format_stamp: "Formatted at 2026-08-06 12:56:36 on dist-test-slave-zkpd"
server_key: "fa16353e1eeb04359e752a1088197b07"
server_key_iv: "e6dc19d46dbbee0eba520125f8399200"
server_key_version: "encryptionkey@0"
I20260806 12:56:36.708171 26024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 12:56:36.708879 26040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:36.709053 26024 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:36.709161 26024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "5e7eb8cc13364a489a1945f4d44e508f"
format_stamp: "Formatted at 2026-08-06 12:56:36 on dist-test-slave-zkpd"
server_key: "fa16353e1eeb04359e752a1088197b07"
server_key_iv: "e6dc19d46dbbee0eba520125f8399200"
server_key_version: "encryptionkey@0"
I20260806 12:56:36.709236 26024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:36.723767 26024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:36.726047 26024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:36.726207 26024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:36.730896 26024 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:33351
I20260806 12:56:36.731014 26102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:33351 every 8 connection(s)
I20260806 12:56:36.731312 26024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:36.732030 26103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:36.734175 26103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f: Bootstrap starting.
I20260806 12:56:36.734695 26103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:36.734921 26103 log.cc:824] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:36.735558 26103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f: No bootstrap required, opened a new log
I20260806 12:56:36.736615 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 26024
I20260806 12:56:36.736711 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:36.736871 24908 external_mini_cluster.cc:1428] Setting key d03c1f1434c12e1fb45f003aa233512d
I20260806 12:56:36.737696 26103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7eb8cc13364a489a1945f4d44e508f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33351 } }
I20260806 12:56:36.737887 26103 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:36.737926 26103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e7eb8cc13364a489a1945f4d44e508f, State: Initialized, Role: FOLLOWER
I20260806 12:56:36.738046 26103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7eb8cc13364a489a1945f4d44e508f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33351 } }
I20260806 12:56:36.738124 26103 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:36.738162 26103 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:36.738210 26103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:36.738936 26103 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7eb8cc13364a489a1945f4d44e508f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33351 } }
I20260806 12:56:36.739053 26103 leader_election.cc:302] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e7eb8cc13364a489a1945f4d44e508f; no voters:
I20260806 12:56:36.739246 26103 leader_election.cc:288] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:36.739275 26108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:36.739398 26108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [term 1 LEADER]: Becoming Leader. State: Replica: 5e7eb8cc13364a489a1945f4d44e508f, State: Running, Role: LEADER
I20260806 12:56:36.739518 26103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:36.739547 26108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7eb8cc13364a489a1945f4d44e508f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33351 } }
I20260806 12:56:36.739920 26108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e7eb8cc13364a489a1945f4d44e508f. Latest consensus state: current_term: 1 leader_uuid: "5e7eb8cc13364a489a1945f4d44e508f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7eb8cc13364a489a1945f4d44e508f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33351 } } }
I20260806 12:56:36.739923 26109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e7eb8cc13364a489a1945f4d44e508f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7eb8cc13364a489a1945f4d44e508f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33351 } } }
I20260806 12:56:36.740067 26108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:36.740267 26109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:36.740317 26117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:36.740816 26117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:36.743665 26117 catalog_manager.cc:1347] Generated new cluster ID: 4d1cd9ccffe94b03bed3d24b276d2ab1
I20260806 12:56:36.743719 26117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:56:36 dist-test-slave-zkpd krb5kdc[26011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:36.748004 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:36.737646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43582 (local address 127.24.83.62:33351)
0806 12:56:36.737824 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:36.737833 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:36.737865 (+ 32us) server_negotiation.cc:407] Connection header received
0806 12:56:36.737995 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:36.738002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:36.738057 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:36.738205 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:36.739093 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:36.740436 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:36.741500 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:36.741931 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:36.742041 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:36.744703 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:36.744707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:36.744707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:36.744719 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:36.746382 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:36.746785 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:36.746788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:36.746788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:36.746895 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:36.747113 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:36.747114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:36.747115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:36.747358 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:36.747534 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:36.747601 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 12:56:36.747737 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260806 12:56:36.768003 26117 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:36.768618 26117 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:36.773974 26117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f: Generated new TSK 0
I20260806 12:56:36.774165 26117 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:56:36.803315 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33351
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:36.886387 26129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:36.886595 26129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:36.886618 26129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:36.887799 26129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:36.887840 26129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:36.887859 26129 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:56:36.887876 26129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:56:36.889350 26129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.83.62:33351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:36.889534 26129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:36.889721 26129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:36.894812 26137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:36.894841 26135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:36.894812 26134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:36.894974 26129 server_base.cc:1034] running on GCE node
I20260806 12:56:36.895310 26129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:36.895568 26129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:36.896747 26129 hybrid_clock.cc:648] HybridClock initialized: now 1786020996896739 us; error 40 us; skew 500 ppm
Aug 06 12:56:36 dist-test-slave-zkpd krb5kdc[26011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020996, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:36.899288 26129 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:56:36.899720 26129 webserver.cc:460] Webserver started at http://127.24.83.1:45341/ using document root <none> and password file <none>
I20260806 12:56:36.900007 26129 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:36.900081 26129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:36.900241 26129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:36.901294 26129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "d0f7be3be5d3431294e18ab9bd0363dc"
format_stamp: "Formatted at 2026-08-06 12:56:36 on dist-test-slave-zkpd"
server_key: "badcf97933556b0af39a5b13b5148c52"
server_key_iv: "a0cc16697f898354775eac5549149a6e"
server_key_version: "encryptionkey@0"
I20260806 12:56:36.901666 26129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "d0f7be3be5d3431294e18ab9bd0363dc"
format_stamp: "Formatted at 2026-08-06 12:56:36 on dist-test-slave-zkpd"
server_key: "badcf97933556b0af39a5b13b5148c52"
server_key_iv: "a0cc16697f898354775eac5549149a6e"
server_key_version: "encryptionkey@0"
I20260806 12:56:36.902987 26129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:56:36.903687 26146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:36.903858 26129 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:56:36.903991 26129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "d0f7be3be5d3431294e18ab9bd0363dc"
format_stamp: "Formatted at 2026-08-06 12:56:36 on dist-test-slave-zkpd"
server_key: "badcf97933556b0af39a5b13b5148c52"
server_key_iv: "a0cc16697f898354775eac5549149a6e"
server_key_version: "encryptionkey@0"
I20260806 12:56:36.904049 26129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:36.914427 26129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:36.916203 26129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:36.916306 26129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:36.916716 26129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:36.916749 26129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:36.916770 26129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:36.916823 26129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:36.923308 26129 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:35563
I20260806 12:56:36.923391 26262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:35563 every 8 connection(s)
Aug 06 12:56:36 dist-test-slave-zkpd krb5kdc[26011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020996, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:36.923693 26129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:56:36.926802 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:36.917372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:58877 (local address 127.24.83.62:33351)
0806 12:56:36.917429 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:36.917431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:36.917626 (+ 195us) server_negotiation.cc:407] Connection header received
0806 12:56:36.918168 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:36.918170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:36.918183 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:36.918244 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:36.919067 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:36.919924 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:36.921420 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:36.921643 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:36.921729 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:36.924215 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:36.924217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:36.924217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:36.924224 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:36.925412 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:36.925820 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:36.925822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:36.925822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:36.925885 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:36.926145 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:36.926146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:36.926146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:36.926260 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:36.926350 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:36.926619 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 12:56:36.926712 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 12:56:36.926957 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:36.917370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33351 (local address 127.24.83.1:58877)
0806 12:56:36.917514 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:36.917546 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:36.918033 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:36.918310 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:36.918314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:36.918662 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:36.918969 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:36.918975 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:36.920005 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:36.920006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:36.921334 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:36.921337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:36.921742 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:36.921743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:36.921891 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:36.922447 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:36.922457 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:36.924135 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:36.925499 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:36.925506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:36.925508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:36.925722 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:36.925954 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:36.925955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:36.925955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:36.926079 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:36.926343 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:36.926349 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:36.926482 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 12:56:36.926580 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 12:56:36.926727 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260806 12:56:36.927464 26263 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33351
I20260806 12:56:36.927558 26263 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:36.927731 26263 heartbeater.cc:511] Master 127.24.83.62:33351 requested a full tablet report, sending...
I20260806 12:56:36.928301 26056 ts_manager.cc:194] Registered new tserver with Master: d0f7be3be5d3431294e18ab9bd0363dc (127.24.83.1:35563)
I20260806 12:56:36.928387 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 26129
I20260806 12:56:36.928488 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:56:36.928710 24908 external_mini_cluster.cc:1428] Setting key 90f6d353197f4120d9b071399f3ea678
I20260806 12:56:36.929167 26056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:58877
I20260806 12:56:36.935029 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:36.929367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:58109 (local address 127.24.83.62:33351)
0806 12:56:36.929421 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:36.929424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:36.929431 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:56:36.929450 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:36.929451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:36.929460 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:36.929516 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:36.929823 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:36.930493 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:36.931169 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:36.931447 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:36.931502 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:36.932541 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:36.932542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:36.932542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:36.932548 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:36.933959 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:36.934282 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:36.934283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:36.934283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:36.934341 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:36.934539 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:36.934540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:36.934541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:36.934640 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:36.934715 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:36.934869 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 12:56:36.934948 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 12:56:36.935036 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:36.928627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33351 (local address 127.24.83.1:58109)
0806 12:56:36.928791 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:36.928801 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:36.928852 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:36.929526 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:36.929527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:36.929699 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:36.929747 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:36.929748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:36.930592 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:36.930593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:36.931089 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:36.931090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:36.931521 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:36.931522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:36.931603 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:36.932063 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:36.932069 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:36.932466 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:36.934043 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:36.934043 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:36.934044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:36.934212 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:36.934411 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:36.934412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:36.934412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:36.934472 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:36.934708 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:36.934709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:36.934754 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 12:56:36.934831 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 12:56:36.934933 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:56:36.954834 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33351
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:37.034716 26272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:37.034948 26272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:37.034988 26272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:37.036166 26272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:37.036232 26272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:37.036267 26272 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:56:37.036331 26272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:56:37.037712 26272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.83.62:33351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:37.037981 26272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:37.038201 26272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:37.043334 26277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:37.043345 26278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:37.043520 26272 server_base.cc:1034] running on GCE node
W20260806 12:56:37.043345 26280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:37.043802 26272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:37.044070 26272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:37.045231 26272 hybrid_clock.cc:648] HybridClock initialized: now 1786020997045221 us; error 27 us; skew 500 ppm
Aug 06 12:56:37 dist-test-slave-zkpd krb5kdc[26011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020997, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:37.047571 26272 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:56:37.048102 26272 webserver.cc:460] Webserver started at http://127.24.83.2:35353/ using document root <none> and password file <none>
I20260806 12:56:37.048321 26272 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:37.048385 26272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:37.048537 26272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:37.049559 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "fd2215bb98be4f96bb7556740480e13c"
format_stamp: "Formatted at 2026-08-06 12:56:37 on dist-test-slave-zkpd"
server_key: "d53d1f9a01ad74b9edbf8aa32c512870"
server_key_iv: "a9f3be76aff76ee71dd7788e44037a54"
server_key_version: "encryptionkey@0"
I20260806 12:56:37.049916 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "fd2215bb98be4f96bb7556740480e13c"
format_stamp: "Formatted at 2026-08-06 12:56:37 on dist-test-slave-zkpd"
server_key: "d53d1f9a01ad74b9edbf8aa32c512870"
server_key_iv: "a9f3be76aff76ee71dd7788e44037a54"
server_key_version: "encryptionkey@0"
I20260806 12:56:37.051234 26272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:37.052042 26289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:37.052237 26272 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:37.052346 26272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "fd2215bb98be4f96bb7556740480e13c"
format_stamp: "Formatted at 2026-08-06 12:56:37 on dist-test-slave-zkpd"
server_key: "d53d1f9a01ad74b9edbf8aa32c512870"
server_key_iv: "a9f3be76aff76ee71dd7788e44037a54"
server_key_version: "encryptionkey@0"
I20260806 12:56:37.052434 26272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:37.071910 26272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:37.073815 26272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:37.073977 26272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:37.074528 26272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:37.074587 26272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:37.074621 26272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:37.074668 26272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:56:37 dist-test-slave-zkpd krb5kdc[26011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020997, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:37.082305 26272 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:44505
I20260806 12:56:37.082378 26406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:44505 every 8 connection(s)
I20260806 12:56:37.082666 26272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:56:37.083992 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.074899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:39501 (local address 127.24.83.62:33351)
0806 12:56:37.074952 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.074954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.075189 (+ 235us) server_negotiation.cc:407] Connection header received
0806 12:56:37.075938 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.075940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.075950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.076001 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:37.076882 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.077378 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.078347 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.078548 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.078625 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.081054 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:37.081056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:37.081056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:37.081063 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:37.082711 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.083203 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.083204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.083204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.083258 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.083443 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.083443 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.083443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.083535 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:37.083593 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.083790 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.083855 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:56:37.084172 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.074863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33351 (local address 127.24.83.2:39501)
0806 12:56:37.075046 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.075094 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.075768 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.076088 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.076092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.076468 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:37.076762 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.076770 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.077480 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.077480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.078253 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.078255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.078634 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.078634 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.078750 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.079264 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:37.079272 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:37.080940 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:37.082820 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:37.082833 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:37.082834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:37.083120 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:37.083316 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:37.083316 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:37.083317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:37.083405 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:37.083570 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:37.083585 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:37.083702 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.083774 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.083881 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260806 12:56:37.084619 26407 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33351
I20260806 12:56:37.084699 26407 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:37.084892 26407 heartbeater.cc:511] Master 127.24.83.62:33351 requested a full tablet report, sending...
I20260806 12:56:37.085233 26057 ts_manager.cc:194] Registered new tserver with Master: fd2215bb98be4f96bb7556740480e13c (127.24.83.2:44505)
I20260806 12:56:37.085660 26057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:39501
I20260806 12:56:37.089187 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 26272
I20260806 12:56:37.089257 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:56:37.089444 24908 external_mini_cluster.cc:1428] Setting key ff1735b02b875e93c795a089067b025a
I20260806 12:56:37.091158 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.085650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:51991 (local address 127.24.83.62:33351)
0806 12:56:37.085692 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.085694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.085833 (+ 139us) server_negotiation.cc:407] Connection header received
0806 12:56:37.085890 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.085891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.085902 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.085951 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:37.086264 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.086726 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.087428 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.087629 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.087713 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.088565 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:37.088566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:37.088567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:37.088574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:37.089972 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.090374 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.090375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.090375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.090425 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.090621 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.090621 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.090621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.090748 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:37.090800 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.090990 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.091072 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 12:56:37.091199 26413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.085587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33351 (local address 127.24.83.2:51991)
0806 12:56:37.085750 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.085760 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.085829 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.085953 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.085954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.086133 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:37.086185 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.086186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.086831 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.086832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.087331 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.087332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.087716 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.087717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.087806 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.088213 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:37.088217 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:37.088473 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:37.090074 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:37.090075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:37.090075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:37.090300 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:37.090489 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:37.090490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:37.090491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:37.090560 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:37.090811 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:37.090812 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:37.090861 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.090932 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.091047 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:56:37.114763 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33351
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:37.194844 26416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:37.195066 26416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:37.195107 26416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:37.196295 26416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:37.196360 26416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:37.196395 26416 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:56:37.196437 26416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:56:37.197887 26416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.83.62:33351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:37.198163 26416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:37.198400 26416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:37.203240 26422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:37.203275 26421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:37.203261 26425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:37.203436 26416 server_base.cc:1034] running on GCE node
I20260806 12:56:37.203747 26416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:37.204030 26416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:37.205174 26416 hybrid_clock.cc:648] HybridClock initialized: now 1786020997205167 us; error 33 us; skew 500 ppm
Aug 06 12:56:37 dist-test-slave-zkpd krb5kdc[26011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020997, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:37.207731 26416 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:56:37.208304 26416 webserver.cc:460] Webserver started at http://127.24.83.3:38561/ using document root <none> and password file <none>
I20260806 12:56:37.208562 26416 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:37.208631 26416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:37.208798 26416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:37.209810 26416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "b299670f91894c27af07585254dd56f4"
format_stamp: "Formatted at 2026-08-06 12:56:37 on dist-test-slave-zkpd"
server_key: "735b4d92b316da23bd6d020d9213bc33"
server_key_iv: "9a6c4347b88c2c3632da7f2c1be23590"
server_key_version: "encryptionkey@0"
I20260806 12:56:37.210178 26416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "b299670f91894c27af07585254dd56f4"
format_stamp: "Formatted at 2026-08-06 12:56:37 on dist-test-slave-zkpd"
server_key: "735b4d92b316da23bd6d020d9213bc33"
server_key_iv: "9a6c4347b88c2c3632da7f2c1be23590"
server_key_version: "encryptionkey@0"
I20260806 12:56:37.211565 26416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:37.212410 26433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:37.212574 26416 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 12:56:37.212672 26416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "b299670f91894c27af07585254dd56f4"
format_stamp: "Formatted at 2026-08-06 12:56:37 on dist-test-slave-zkpd"
server_key: "735b4d92b316da23bd6d020d9213bc33"
server_key_iv: "9a6c4347b88c2c3632da7f2c1be23590"
server_key_version: "encryptionkey@0"
I20260806 12:56:37.212754 26416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:37.223217 26416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:37.225541 26416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:37.225703 26416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:37.226195 26416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:37.226259 26416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:37.226310 26416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:37.226336 26416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:56:37 dist-test-slave-zkpd krb5kdc[26011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020997, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:37.233773 26416 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:45777
I20260806 12:56:37.233850 26549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:45777 every 8 connection(s)
I20260806 12:56:37.234161 26416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:56:37.236603 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.226637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:39341 (local address 127.24.83.62:33351)
0806 12:56:37.226711 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.226714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.226984 (+ 270us) server_negotiation.cc:407] Connection header received
0806 12:56:37.227675 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.227676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.227689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.227755 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:37.228723 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.229465 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.231111 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.231430 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.231507 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.234042 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:37.234044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:37.234045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:37.234051 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:37.235476 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.235817 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.235818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.235818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.235865 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.236124 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.236125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.236125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.236222 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:37.236272 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.236490 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.236545 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:56:37.236749 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.226617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33351 (local address 127.24.83.3:39341)
0806 12:56:37.226816 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.226868 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.227557 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.227847 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.227852 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.228282 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:37.228593 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.228601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.229581 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.229582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.231008 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.231012 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.231538 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.231540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.231696 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.232218 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:37.232229 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:37.233951 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:37.235549 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:37.235554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:37.235556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:37.235753 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:37.235968 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:37.235968 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:37.235969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:37.236069 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:37.236277 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:37.236281 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:37.236388 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.236465 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.236574 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260806 12:56:37.237202 26550 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33351
I20260806 12:56:37.237282 26550 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:37.237478 26550 heartbeater.cc:511] Master 127.24.83.62:33351 requested a full tablet report, sending...
I20260806 12:56:37.237927 26056 ts_manager.cc:194] Registered new tserver with Master: b299670f91894c27af07585254dd56f4 (127.24.83.3:45777)
I20260806 12:56:37.238467 26056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:39341
I20260806 12:56:37.239218 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 26416
I20260806 12:56:37.239351 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:56:37.239517 24908 external_mini_cluster.cc:1428] Setting key 597167b8993cf00997472827b8399619
I20260806 12:56:37.240556 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:56:37.244567 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.238354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:44613 (local address 127.24.83.62:33351)
0806 12:56:37.238412 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.238415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.238450 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:56:37.238522 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.238523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.238532 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.238577 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:37.238930 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.239650 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.240685 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.240911 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.240975 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.241975 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:37.241976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:37.241976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:37.241982 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:37.243590 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.243884 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.243885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.243885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.243993 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.244181 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.244182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.244182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.244264 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:37.244316 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.244463 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.244514 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 12:56:37.244696 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.238244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33351 (local address 127.24.83.3:44613)
0806 12:56:37.238393 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.238404 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.238466 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.238615 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.238616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.238805 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:37.238866 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.238868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.239768 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.239769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.240586 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.240589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.241004 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.241006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.241089 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.241464 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:37.241469 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:37.241880 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:37.243675 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:37.243676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:37.243676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:37.243828 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:37.244056 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:37.244057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:37.244057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:37.244122 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:37.244322 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:37.244323 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:37.244369 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.244448 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.244556 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
Aug 06 12:56:37 dist-test-slave-zkpd krb5kdc[26011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020997, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 12:56:37 dist-test-slave-zkpd krb5kdc[26011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020997, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:37.263870 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.257605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43588 (local address 127.24.83.62:33351)
0806 12:56:37.257664 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.257666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.257674 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:56:37.257695 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.257695 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.257704 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.257775 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:37.258314 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.258767 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.259475 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.259658 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.259759 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.261849 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:37.261851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:37.261851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:37.261857 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:37.262962 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.263261 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.263262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.263262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.263314 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.263471 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.263471 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.263472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.263556 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:37.263625 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.263704 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.263756 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 12:56:37.265718 26056 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:56:37.267194 26056 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:56:37.273320 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.269274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:45777 (local address 127.0.0.1:56910)
0806 12:56:37.269566 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.269606 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.269680 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.269921 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.269924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.269926 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:37.269991 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.269993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.271290 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.271290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.272284 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.272287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.273114 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.273115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.273170 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.273180 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.273233 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.273268 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":61,"threads_started":1}
I20260806 12:56:37.273381 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.269367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56910 (local address 127.24.83.3:45777)
0806 12:56:37.269524 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.269530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.269660 (+ 130us) server_negotiation.cc:407] Connection header received
0806 12:56:37.269758 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.269760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.269774 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.269848 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:37.270074 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.271198 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.272372 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.273034 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.273107 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.273161 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.273254 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.273286 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260806 12:56:37.273805 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.269274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:44505 (local address 127.0.0.1:44684)
0806 12:56:37.269630 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.269643 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.269694 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.270201 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.270202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.270204 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:37.270248 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.270250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.271228 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.271229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.272558 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.272561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.273552 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.273553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.273618 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.273622 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.273694 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.273734 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":62,"thread_start_us":61,"threads_started":1}
I20260806 12:56:37.273844 26568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.269449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.24.83.2:44505)
0806 12:56:37.269905 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.269910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.269919 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:56:37.269963 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.269964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.269977 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.270069 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:37.270337 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.271137 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.272648 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.273382 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.273453 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.273515 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.273709 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.273748 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":93,"threads_started":1}
I20260806 12:56:37.274055 26484 tablet_service.cc:1467] Processing CreateTablet for tablet 81ce4b4a701f4c3b845c8aea1f84e0c7 (DEFAULT_TABLE table=test-table [id=ae354f21c20b49c3be0e8861a460d9be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:37.274456 26484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ce4b4a701f4c3b845c8aea1f84e0c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:37.274459 26340 tablet_service.cc:1467] Processing CreateTablet for tablet 81ce4b4a701f4c3b845c8aea1f84e0c7 (DEFAULT_TABLE table=test-table [id=ae354f21c20b49c3be0e8861a460d9be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:37.274801 26340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ce4b4a701f4c3b845c8aea1f84e0c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:37.275282 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.269470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:35563 (local address 127.0.0.1:53734)
0806 12:56:37.269781 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.269804 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.269844 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.270033 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.270034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.270036 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:37.270090 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.270091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.271389 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.271389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.272558 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.272561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.275043 (+ 2482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.275044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.275107 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.275111 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.275173 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.275212 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":9,"thread_start_us":51,"threads_started":1}
I20260806 12:56:37.275331 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.269689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53734 (local address 127.24.83.1:35563)
0806 12:56:37.269859 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.269864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.269872 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:56:37.269922 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.269924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.269939 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.270023 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:37.270156 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.271310 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.273994 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.274953 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.275026 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.275077 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.275194 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.275233 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260806 12:56:37.276192 26197 tablet_service.cc:1467] Processing CreateTablet for tablet 81ce4b4a701f4c3b845c8aea1f84e0c7 (DEFAULT_TABLE table=test-table [id=ae354f21c20b49c3be0e8861a460d9be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:37.276542 26197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ce4b4a701f4c3b845c8aea1f84e0c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:37.277845 26570 tablet_bootstrap.cc:492] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: Bootstrap starting.
I20260806 12:56:37.277976 26571 tablet_bootstrap.cc:492] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: Bootstrap starting.
I20260806 12:56:37.278478 26570 tablet_bootstrap.cc:654] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:37.278702 26570 log.cc:824] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:37.279307 26571 tablet_bootstrap.cc:654] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:37.279335 26570 tablet_bootstrap.cc:492] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: No bootstrap required, opened a new log
I20260806 12:56:37.279378 26570 ts_tablet_manager.cc:1397] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:56:37.280289 26571 log.cc:824] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:37.280745 26573 tablet_bootstrap.cc:492] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: Bootstrap starting.
I20260806 12:56:37.281133 26571 tablet_bootstrap.cc:492] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: No bootstrap required, opened a new log
I20260806 12:56:37.281216 26571 ts_tablet_manager.cc:1397] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260806 12:56:37.281441 26573 tablet_bootstrap.cc:654] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:37.282097 26573 log.cc:824] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:37.283078 26573 tablet_bootstrap.cc:492] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: No bootstrap required, opened a new log
I20260806 12:56:37.282972 26570 raft_consensus.cc:348] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.283131 26570 raft_consensus.cc:374] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:37.283145 26573 ts_tablet_manager.cc:1397] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:56:37.283170 26570 raft_consensus.cc:729] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2215bb98be4f96bb7556740480e13c, State: Initialized, Role: FOLLOWER
I20260806 12:56:37.283372 26570 consensus_queue.cc:260] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.283768 26570 ts_tablet_manager.cc:1428] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260806 12:56:37.283978 26407 heartbeater.cc:503] Master 127.24.83.62:33351 was elected leader, sending a full tablet report...
I20260806 12:56:37.284643 26571 raft_consensus.cc:348] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.284778 26571 raft_consensus.cc:374] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:37.284840 26571 raft_consensus.cc:729] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b299670f91894c27af07585254dd56f4, State: Initialized, Role: FOLLOWER
I20260806 12:56:37.284955 26571 consensus_queue.cc:260] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.285379 26571 ts_tablet_manager.cc:1428] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 12:56:37.285413 26550 heartbeater.cc:503] Master 127.24.83.62:33351 was elected leader, sending a full tablet report...
I20260806 12:56:37.286866 26573 raft_consensus.cc:348] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.286985 26573 raft_consensus.cc:374] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:37.287006 26573 raft_consensus.cc:729] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f7be3be5d3431294e18ab9bd0363dc, State: Initialized, Role: FOLLOWER
I20260806 12:56:37.287077 26573 consensus_queue.cc:260] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.287320 26573 ts_tablet_manager.cc:1428] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260806 12:56:37.287438 26263 heartbeater.cc:503] Master 127.24.83.62:33351 was elected leader, sending a full tablet report...
W20260806 12:56:37.333392 26408 tablet.cc:2369] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:56:37.424589 26264 tablet.cc:2369] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:56:37.484730 26551 tablet.cc:2369] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:56:37.616889 26577 raft_consensus.cc:482] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:56:37.617018 26577 raft_consensus.cc:504] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.617411 26577 leader_election.cc:288] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd2215bb98be4f96bb7556740480e13c (127.24.83.2:44505), d0f7be3be5d3431294e18ab9bd0363dc (127.24.83.1:35563)
I20260806 12:56:37.621536 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.617931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:54345 (local address 127.24.83.1:35563)
0806 12:56:37.617988 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.617990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.618001 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:56:37.618079 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.618079 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.618087 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.618124 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:37.618470 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.619150 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.620485 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.621029 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.621105 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.621115 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.621422 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.621464 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:56:37.621537 26568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.617657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:37573 (local address 127.24.83.2:44505)
0806 12:56:37.617728 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.617730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.617757 (+ 27us) server_negotiation.cc:407] Connection header received
0806 12:56:37.617862 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.617864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.617876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.617931 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:37.618297 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.619017 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.620486 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.621027 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.621105 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.621115 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.621444 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.621478 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 12:56:37.621526 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.617618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:44505 (local address 127.24.83.3:37573)
0806 12:56:37.617688 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.617701 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.617776 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.617966 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.617968 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.617970 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:37.618186 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.618188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.619101 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.619101 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.620392 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.620395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.621115 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.621116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.621369 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.621373 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.621426 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.621456 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":2}
I20260806 12:56:37.621546 26579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.617761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:35563 (local address 127.24.83.3:54345)
0806 12:56:37.617955 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.617965 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.618016 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.618139 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.618139 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.618140 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:37.618396 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.618398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.619217 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.619218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.620393 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.620395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.621115 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.621116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.621282 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.621287 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.621384 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.621435 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":3,"thread_start_us":135,"threads_started":1}
I20260806 12:56:37.622062 26360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ce4b4a701f4c3b845c8aea1f84e0c7" candidate_uuid: "b299670f91894c27af07585254dd56f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2215bb98be4f96bb7556740480e13c" is_pre_election: true
I20260806 12:56:37.622062 26217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ce4b4a701f4c3b845c8aea1f84e0c7" candidate_uuid: "b299670f91894c27af07585254dd56f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" is_pre_election: true
I20260806 12:56:37.622226 26360 raft_consensus.cc:2454] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b299670f91894c27af07585254dd56f4 in term 0.
I20260806 12:56:37.622226 26217 raft_consensus.cc:2454] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b299670f91894c27af07585254dd56f4 in term 0.
I20260806 12:56:37.622512 26436 leader_election.cc:302] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b299670f91894c27af07585254dd56f4, d0f7be3be5d3431294e18ab9bd0363dc; no voters:
I20260806 12:56:37.622655 26577 raft_consensus.cc:2781] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:56:37.622742 26577 raft_consensus.cc:482] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:56:37.622779 26577 raft_consensus.cc:3037] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:37.624217 26577 raft_consensus.cc:504] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.624364 26577 leader_election.cc:288] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [CANDIDATE]: Term 1 election: Requested vote from peers fd2215bb98be4f96bb7556740480e13c (127.24.83.2:44505), d0f7be3be5d3431294e18ab9bd0363dc (127.24.83.1:35563)
I20260806 12:56:37.624512 26360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ce4b4a701f4c3b845c8aea1f84e0c7" candidate_uuid: "b299670f91894c27af07585254dd56f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2215bb98be4f96bb7556740480e13c"
I20260806 12:56:37.624555 26217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ce4b4a701f4c3b845c8aea1f84e0c7" candidate_uuid: "b299670f91894c27af07585254dd56f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f7be3be5d3431294e18ab9bd0363dc"
I20260806 12:56:37.624603 26360 raft_consensus.cc:3037] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:37.624626 26217 raft_consensus.cc:3037] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:37.625656 26360 raft_consensus.cc:2454] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b299670f91894c27af07585254dd56f4 in term 1.
I20260806 12:56:37.625658 26217 raft_consensus.cc:2454] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b299670f91894c27af07585254dd56f4 in term 1.
I20260806 12:56:37.625839 26436 leader_election.cc:302] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b299670f91894c27af07585254dd56f4, fd2215bb98be4f96bb7556740480e13c; no voters:
I20260806 12:56:37.625958 26577 raft_consensus.cc:2781] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:37.626114 26577 raft_consensus.cc:686] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 1 LEADER]: Becoming Leader. State: Replica: b299670f91894c27af07585254dd56f4, State: Running, Role: LEADER
I20260806 12:56:37.626219 26577 consensus_queue.cc:237] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.627059 26056 catalog_manager.cc:5495] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b299670f91894c27af07585254dd56f4 (127.24.83.3). New cstate: current_term: 1 leader_uuid: "b299670f91894c27af07585254dd56f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } health_report { overall_health: HEALTHY } } }
I20260806 12:56:37.717010 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.714253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56916 (local address 127.24.83.3:45777)
0806 12:56:37.714334 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.714336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.714369 (+ 33us) server_negotiation.cc:407] Connection header received
0806 12:56:37.714473 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.714473 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.714492 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.714553 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:37.714681 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.715192 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.716050 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.716233 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.716288 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.716561 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:37.716653 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:37.716712 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.716757 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.716776 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 12:56:37.719096 26360 raft_consensus.cc:1261] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c [term 1 FOLLOWER]: Refusing update from remote peer b299670f91894c27af07585254dd56f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:37.719097 26217 raft_consensus.cc:1261] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc [term 1 FOLLOWER]: Refusing update from remote peer b299670f91894c27af07585254dd56f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:37.719571 26580 consensus_queue.cc:1035] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:37.719699 26577 consensus_queue.cc:1035] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:37.722301 26583 mvcc.cc:205] Tried to move back new op lower bound from 7315542006654361600 to 7315542006277484544. Current Snapshot: MvccSnapshot[applied={T|T < 7315542006654361600}]
I20260806 12:56:37.723280 26585 mvcc.cc:205] Tried to move back new op lower bound from 7315542006654361600 to 7315542006277484544. Current Snapshot: MvccSnapshot[applied={T|T < 7315542006654361600 or (T in {7315542006654361600})}]
I20260806 12:56:37.727437 26057 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:37.727568 26057 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:37.728425 26057 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f: Sending DeleteTablet for 3 replicas of tablet 81ce4b4a701f4c3b845c8aea1f84e0c7
I20260806 12:56:37.728837 26484 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ce4b4a701f4c3b845c8aea1f84e0c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:37 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56910
I20260806 12:56:37.729033 26197 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ce4b4a701f4c3b845c8aea1f84e0c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:37 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:53734
I20260806 12:56:37.729363 26594 tablet_replica.cc:330] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: stopping tablet replica
I20260806 12:56:37.729574 26594 raft_consensus.cc:2229] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:56:37.729560 26057 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 12:56:37.729724 26594 raft_consensus.cc:2258] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 12:56:37.729725 26057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:56:37.730896 26197 tablet_service.cc:1467] Processing CreateTablet for tablet 52f758d777454f5a804a246192bb2eb8 (DEFAULT_TABLE table=test-table [id=255ec83dde0c44e584851f65108e8ee0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:37.730880 26340 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ce4b4a701f4c3b845c8aea1f84e0c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:37 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:44684
I20260806 12:56:37.731027 26197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52f758d777454f5a804a246192bb2eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:37.731194 26596 tablet_replica.cc:330] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: stopping tablet replica
I20260806 12:56:37.731285 26596 raft_consensus.cc:2229] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:37.731374 26596 raft_consensus.cc:2258] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:37.731547 26595 tablet_replica.cc:330] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: stopping tablet replica
I20260806 12:56:37.731635 26595 raft_consensus.cc:2229] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:37.731724 26595 raft_consensus.cc:2258] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:37.732373 26596 ts_tablet_manager.cc:1905] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:37.733011 26484 tablet_service.cc:1467] Processing CreateTablet for tablet 52f758d777454f5a804a246192bb2eb8 (DEFAULT_TABLE table=test-table [id=255ec83dde0c44e584851f65108e8ee0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:37.733150 26484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52f758d777454f5a804a246192bb2eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:37.735013 26595 ts_tablet_manager.cc:1905] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:37.735091 26573 tablet_bootstrap.cc:492] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc: Bootstrap starting.
I20260806 12:56:37.735087 26571 tablet_bootstrap.cc:492] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4: Bootstrap starting.
I20260806 12:56:37.735860 26573 tablet_bootstrap.cc:654] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:37.736021 26571 tablet_bootstrap.cc:654] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:37.736996 26340 tablet_service.cc:1467] Processing CreateTablet for tablet 52f758d777454f5a804a246192bb2eb8 (DEFAULT_TABLE table=test-table [id=255ec83dde0c44e584851f65108e8ee0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:37.737764 26340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52f758d777454f5a804a246192bb2eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:37.737607 26595 ts_tablet_manager.cc:1918] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:56:37.737845 26595 log.cc:1196] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/wal/wals/81ce4b4a701f4c3b845c8aea1f84e0c7
I20260806 12:56:37.738152 26595 ts_tablet_manager.cc:1939] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P d0f7be3be5d3431294e18ab9bd0363dc: Deleting consensus metadata
I20260806 12:56:37.737764 26596 ts_tablet_manager.cc:1918] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:56:37.738739 26596 log.cc:1196] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/81ce4b4a701f4c3b845c8aea1f84e0c7
I20260806 12:56:37.738730 26043 catalog_manager.cc:4841] TS d0f7be3be5d3431294e18ab9bd0363dc (127.24.83.1:35563): tablet 81ce4b4a701f4c3b845c8aea1f84e0c7 (table test-table [id=ae354f21c20b49c3be0e8861a460d9be]) successfully deleted
I20260806 12:56:37.739017 26596 ts_tablet_manager.cc:1939] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P fd2215bb98be4f96bb7556740480e13c: Deleting consensus metadata
I20260806 12:56:37.739493 26043 catalog_manager.cc:4841] TS fd2215bb98be4f96bb7556740480e13c (127.24.83.2:44505): tablet 81ce4b4a701f4c3b845c8aea1f84e0c7 (table test-table [id=ae354f21c20b49c3be0e8861a460d9be]) successfully deleted
I20260806 12:56:37.739645 26570 tablet_bootstrap.cc:492] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c: Bootstrap starting.
I20260806 12:56:37.739907 26594 ts_tablet_manager.cc:1905] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:37.740401 26570 tablet_bootstrap.cc:654] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:37.740397 26573 tablet_bootstrap.cc:492] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc: No bootstrap required, opened a new log
I20260806 12:56:37.740464 26573 ts_tablet_manager.cc:1397] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 12:56:37.740664 26573 raft_consensus.cc:348] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.740841 26573 raft_consensus.cc:374] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:37.740877 26573 raft_consensus.cc:729] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f7be3be5d3431294e18ab9bd0363dc, State: Initialized, Role: FOLLOWER
I20260806 12:56:37.740996 26573 consensus_queue.cc:260] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.741163 26573 ts_tablet_manager.cc:1428] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:37.741586 26570 tablet_bootstrap.cc:492] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c: No bootstrap required, opened a new log
I20260806 12:56:37.741647 26570 ts_tablet_manager.cc:1397] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:56:37.742024 26570 raft_consensus.cc:348] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.742125 26570 raft_consensus.cc:374] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:37.742156 26570 raft_consensus.cc:729] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2215bb98be4f96bb7556740480e13c, State: Initialized, Role: FOLLOWER
I20260806 12:56:37.742211 26570 consensus_queue.cc:260] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.742434 26594 ts_tablet_manager.cc:1918] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:56:37.742496 26594 log.cc:1196] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/81ce4b4a701f4c3b845c8aea1f84e0c7
I20260806 12:56:37.742627 26570 ts_tablet_manager.cc:1428] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 12:56:37.742774 26594 ts_tablet_manager.cc:1939] T 81ce4b4a701f4c3b845c8aea1f84e0c7 P b299670f91894c27af07585254dd56f4: Deleting consensus metadata
I20260806 12:56:37.743155 26042 catalog_manager.cc:4841] TS b299670f91894c27af07585254dd56f4 (127.24.83.3:45777): tablet 81ce4b4a701f4c3b845c8aea1f84e0c7 (table test-table [id=ae354f21c20b49c3be0e8861a460d9be]) successfully deleted
I20260806 12:56:37.752350 26571 tablet_bootstrap.cc:492] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4: No bootstrap required, opened a new log
I20260806 12:56:37.752419 26571 ts_tablet_manager.cc:1397] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20260806 12:56:37.752763 26571 raft_consensus.cc:348] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.752852 26571 raft_consensus.cc:374] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:37.752874 26571 raft_consensus.cc:729] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b299670f91894c27af07585254dd56f4, State: Initialized, Role: FOLLOWER
I20260806 12:56:37.752918 26571 consensus_queue.cc:260] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.753010 26571 ts_tablet_manager.cc:1428] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 12:56:37.829748 26578 raft_consensus.cc:482] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:56:37.829864 26578 raft_consensus.cc:504] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.830237 26578 leader_election.cc:288] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd2215bb98be4f96bb7556740480e13c (127.24.83.2:44505), b299670f91894c27af07585254dd56f4 (127.24.83.3:45777)
I20260806 12:56:37.835374 26568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.830691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:49605 (local address 127.24.83.2:44505)
0806 12:56:37.830739 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.830742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.830974 (+ 232us) server_negotiation.cc:407] Connection header received
0806 12:56:37.831087 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.831088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.831098 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.831129 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:37.831411 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.832153 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.834076 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.834709 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.834761 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.834775 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.835231 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.835289 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:56:37.835445 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.830667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:49369 (local address 127.24.83.3:45777)
0806 12:56:37.830721 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.830724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.830873 (+ 149us) server_negotiation.cc:407] Connection header received
0806 12:56:37.830991 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.830992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.831002 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.831036 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:37.831352 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.832226 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.834142 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.834821 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.834869 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.834881 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.835298 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.835345 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 12:56:37.835721 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.830424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:45777 (local address 127.24.83.1:49369)
0806 12:56:37.830803 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.830816 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.830907 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.831110 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.831111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.831113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:37.831267 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.831270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.832338 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.832339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.834022 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.834026 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.835033 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.835034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.835199 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.835205 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.835533 (+ 328us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.835588 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":61,"threads_started":1}
I20260806 12:56:37.836154 26504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f758d777454f5a804a246192bb2eb8" candidate_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b299670f91894c27af07585254dd56f4" is_pre_election: true
I20260806 12:56:37.836345 26504 raft_consensus.cc:2454] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f7be3be5d3431294e18ab9bd0363dc in term 0.
I20260806 12:56:37.836400 26601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.830463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:44505 (local address 127.24.83.1:49605)
0806 12:56:37.830901 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.830912 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.831005 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.831207 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.831208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.831210 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:37.831326 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.831328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.832266 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.832267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.833838 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.833842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.834957 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.834959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.835100 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.835113 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.836126 (+ 1013us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.836193 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":75,"thread_start_us":115,"threads_started":1}
I20260806 12:56:37.836656 26360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f758d777454f5a804a246192bb2eb8" candidate_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2215bb98be4f96bb7556740480e13c" is_pre_election: true
I20260806 12:56:37.836642 26148 leader_election.cc:302] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b299670f91894c27af07585254dd56f4, d0f7be3be5d3431294e18ab9bd0363dc; no voters:
I20260806 12:56:37.836745 26360 raft_consensus.cc:2454] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f7be3be5d3431294e18ab9bd0363dc in term 0.
I20260806 12:56:37.836791 26578 raft_consensus.cc:2781] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:56:37.836830 26578 raft_consensus.cc:482] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:56:37.836854 26578 raft_consensus.cc:3037] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:37.838479 26578 raft_consensus.cc:504] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.838631 26578 leader_election.cc:288] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [CANDIDATE]: Term 1 election: Requested vote from peers fd2215bb98be4f96bb7556740480e13c (127.24.83.2:44505), b299670f91894c27af07585254dd56f4 (127.24.83.3:45777)
I20260806 12:56:37.838867 26360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f758d777454f5a804a246192bb2eb8" candidate_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2215bb98be4f96bb7556740480e13c"
I20260806 12:56:37.838971 26360 raft_consensus.cc:3037] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:37.839956 26504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f758d777454f5a804a246192bb2eb8" candidate_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b299670f91894c27af07585254dd56f4"
I20260806 12:56:37.840121 26504 raft_consensus.cc:3037] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:37.845625 26504 raft_consensus.cc:2454] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f7be3be5d3431294e18ab9bd0363dc in term 1.
I20260806 12:56:37.845880 26148 leader_election.cc:302] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b299670f91894c27af07585254dd56f4, d0f7be3be5d3431294e18ab9bd0363dc; no voters:
I20260806 12:56:37.846716 26360 raft_consensus.cc:2454] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f7be3be5d3431294e18ab9bd0363dc in term 1.
I20260806 12:56:37.846980 26578 raft_consensus.cc:2781] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:37.847172 26578 raft_consensus.cc:686] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 1 LEADER]: Becoming Leader. State: Replica: d0f7be3be5d3431294e18ab9bd0363dc, State: Running, Role: LEADER
I20260806 12:56:37.847278 26578 consensus_queue.cc:237] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.848268 26057 catalog_manager.cc:5495] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc reported cstate change: term changed from 0 to 1, leader changed from <none> to d0f7be3be5d3431294e18ab9bd0363dc (127.24.83.1). New cstate: current_term: 1 leader_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } health_report { overall_health: UNKNOWN } } }
Aug 06 12:56:37 dist-test-slave-zkpd krb5kdc[26011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020996, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:37.937100 26610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.925223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43594 (local address 127.24.83.62:33351)
0806 12:56:37.925582 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.925585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.925598 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:56:37.925680 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.925682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.925695 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.925781 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:37.926425 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.927134 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.927848 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.928083 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.928137 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.934550 (+ 6413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:37.934551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:37.934551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:37.934557 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:37.936012 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.936358 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.936358 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.936359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.936411 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:37.936572 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:37.936573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:37.936573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:37.936674 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:37.936724 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.936854 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.936971 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":190,"threads_started":1}
I20260806 12:56:37.937109 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.925157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33351 (local address 127.0.0.1:43594)
0806 12:56:37.925517 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.925529 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.925602 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.925806 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.925807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.926255 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:37.926325 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.926327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.927211 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.927212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.927782 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.927784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.928146 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.928147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.928222 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.933122 (+ 4900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:37.933126 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:37.934472 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:37.936087 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:37.936094 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:37.936096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:37.936297 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:37.936469 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:37.936470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:37.936470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:37.936523 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:37.936727 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:37.936732 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:37.936772 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.936823 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.937026 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":88,"thread_start_us":277,"threads_started":1}
I20260806 12:56:37.939358 26057 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:43594:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 12:56:37.939594 26057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:56:37.940995 26197 tablet_service.cc:1467] Processing CreateTablet for tablet 1024a99f3d2c4670be838054e0eb1892 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb5d7c054eb4f2ba688fee5219f7a5f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:56:37.941093 26484 tablet_service.cc:1467] Processing CreateTablet for tablet 1024a99f3d2c4670be838054e0eb1892 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb5d7c054eb4f2ba688fee5219f7a5f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:56:37.941241 26197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1024a99f3d2c4670be838054e0eb1892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:37.941249 26484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1024a99f3d2c4670be838054e0eb1892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:37.941284 26340 tablet_service.cc:1467] Processing CreateTablet for tablet 1024a99f3d2c4670be838054e0eb1892 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb5d7c054eb4f2ba688fee5219f7a5f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:56:37.941459 26340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1024a99f3d2c4670be838054e0eb1892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:37.942705 26571 tablet_bootstrap.cc:492] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: Bootstrap starting.
I20260806 12:56:37.943015 26571 tablet_bootstrap.cc:654] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:37.943234 26573 tablet_bootstrap.cc:492] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc: Bootstrap starting.
I20260806 12:56:37.943291 26570 tablet_bootstrap.cc:492] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c: Bootstrap starting.
I20260806 12:56:37.943540 26573 tablet_bootstrap.cc:654] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:37.944218 26573 tablet_bootstrap.cc:492] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc: No bootstrap required, opened a new log
I20260806 12:56:37.944274 26573 ts_tablet_manager.cc:1397] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:56:37.944387 26570 tablet_bootstrap.cc:654] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:37.944444 26573 raft_consensus.cc:348] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.944520 26573 raft_consensus.cc:374] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:37.944552 26573 raft_consensus.cc:729] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f7be3be5d3431294e18ab9bd0363dc, State: Initialized, Role: FOLLOWER
I20260806 12:56:37.944595 26573 consensus_queue.cc:260] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.944708 26573 ts_tablet_manager.cc:1428] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:37.944789 26602 tablet_replica.cc:439] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } } }
I20260806 12:56:37.944880 26602 tablet_replica.cc:442] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:37.944934 26571 tablet_bootstrap.cc:492] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: No bootstrap required, opened a new log
I20260806 12:56:37.944980 26571 ts_tablet_manager.cc:1397] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 12:56:37.945099 26571 raft_consensus.cc:348] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.945175 26571 raft_consensus.cc:374] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:37.945211 26571 raft_consensus.cc:729] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b299670f91894c27af07585254dd56f4, State: Initialized, Role: FOLLOWER
I20260806 12:56:37.945266 26571 consensus_queue.cc:260] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.945392 26571 ts_tablet_manager.cc:1428] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:37.945400 26570 tablet_bootstrap.cc:492] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c: No bootstrap required, opened a new log
I20260806 12:56:37.945415 26588 tablet_replica.cc:439] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } } }
I20260806 12:56:37.945468 26570 ts_tablet_manager.cc:1397] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 12:56:37.945492 26588 tablet_replica.cc:442] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:37.945736 26570 raft_consensus.cc:348] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.945823 26570 raft_consensus.cc:374] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:37.945853 26570 raft_consensus.cc:729] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2215bb98be4f96bb7556740480e13c, State: Initialized, Role: FOLLOWER
I20260806 12:56:37.945894 26570 consensus_queue.cc:260] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.945998 26570 ts_tablet_manager.cc:1428] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:37.946077 26576 tablet_replica.cc:439] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } } }
I20260806 12:56:37.946201 26576 tablet_replica.cc:442] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:37.950829 26588 raft_consensus.cc:482] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:56:37.950887 26588 raft_consensus.cc:504] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.950973 26588 leader_election.cc:288] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0f7be3be5d3431294e18ab9bd0363dc (127.24.83.1:35563), fd2215bb98be4f96bb7556740480e13c (127.24.83.2:44505)
I20260806 12:56:37.951145 26360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1024a99f3d2c4670be838054e0eb1892" candidate_uuid: "b299670f91894c27af07585254dd56f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2215bb98be4f96bb7556740480e13c" is_pre_election: true
I20260806 12:56:37.951226 26360 raft_consensus.cc:2454] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b299670f91894c27af07585254dd56f4 in term 0.
I20260806 12:56:37.951234 26217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1024a99f3d2c4670be838054e0eb1892" candidate_uuid: "b299670f91894c27af07585254dd56f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" is_pre_election: true
I20260806 12:56:37.951339 26217 raft_consensus.cc:2454] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b299670f91894c27af07585254dd56f4 in term 0.
I20260806 12:56:37.951359 26436 leader_election.cc:302] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b299670f91894c27af07585254dd56f4, fd2215bb98be4f96bb7556740480e13c; no voters:
I20260806 12:56:37.951433 26588 raft_consensus.cc:2781] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:56:37.951481 26588 raft_consensus.cc:482] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:56:37.951501 26588 raft_consensus.cc:3037] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:37.952167 26588 raft_consensus.cc:504] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.952286 26588 leader_election.cc:288] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [CANDIDATE]: Term 1 election: Requested vote from peers d0f7be3be5d3431294e18ab9bd0363dc (127.24.83.1:35563), fd2215bb98be4f96bb7556740480e13c (127.24.83.2:44505)
I20260806 12:56:37.952426 26217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1024a99f3d2c4670be838054e0eb1892" candidate_uuid: "b299670f91894c27af07585254dd56f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f7be3be5d3431294e18ab9bd0363dc"
I20260806 12:56:37.952497 26217 raft_consensus.cc:3037] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:37.952459 26360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1024a99f3d2c4670be838054e0eb1892" candidate_uuid: "b299670f91894c27af07585254dd56f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2215bb98be4f96bb7556740480e13c"
I20260806 12:56:37.952545 26360 raft_consensus.cc:3037] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:37.953030 26360 raft_consensus.cc:2454] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b299670f91894c27af07585254dd56f4 in term 1.
I20260806 12:56:37.953194 26436 leader_election.cc:302] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b299670f91894c27af07585254dd56f4, fd2215bb98be4f96bb7556740480e13c; no voters:
I20260806 12:56:37.953238 26217 raft_consensus.cc:2454] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b299670f91894c27af07585254dd56f4 in term 1.
I20260806 12:56:37.953347 26588 raft_consensus.cc:2781] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:37.953408 26588 raft_consensus.cc:686] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [term 1 LEADER]: Becoming Leader. State: Replica: b299670f91894c27af07585254dd56f4, State: Running, Role: LEADER
I20260806 12:56:37.953461 26588 consensus_queue.cc:237] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } }
I20260806 12:56:37.953709 26577 tablet_replica.cc:439] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: TxnStatusTablet state changed. Reason: New leader b299670f91894c27af07585254dd56f4. Latest consensus state: current_term: 1 leader_uuid: "b299670f91894c27af07585254dd56f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } } }
I20260806 12:56:37.953802 26577 tablet_replica.cc:442] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:56:37.954013 26056 catalog_manager.cc:5495] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b299670f91894c27af07585254dd56f4 (127.24.83.3). New cstate: current_term: 1 leader_uuid: "b299670f91894c27af07585254dd56f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } health_report { overall_health: HEALTHY } } }
I20260806 12:56:37.954295 26617 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 12:56:37.954425 26617 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 12:56:37.962860 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.960415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.24.83.3:45777)
0806 12:56:37.960470 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.960471 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.960479 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:56:37.960497 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.960498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.960509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.960556 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:37.960711 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.961167 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.962225 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.962407 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.962461 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.962637 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:37.962661 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:37.962695 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.962794 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.962812 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 12:56:37.962916 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.960170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:45777 (local address 127.0.0.1:56924)
0806 12:56:37.960249 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.960262 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.960313 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.960588 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.960589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.960591 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:56:37.960640 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.960643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.961240 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.961241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.962142 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.962145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.962479 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.962480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.962538 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.962555 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:56:37.962715 (+ 160us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:56:37.962723 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.962788 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.962842 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 12:56:37.963881 26217 raft_consensus.cc:1261] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc [term 1 FOLLOWER]: Refusing update from remote peer b299670f91894c27af07585254dd56f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:37.963929 26360 raft_consensus.cc:1261] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c [term 1 FOLLOWER]: Refusing update from remote peer b299670f91894c27af07585254dd56f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:37.964111 26588 consensus_queue.cc:1035] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:37.964207 26577 consensus_queue.cc:1035] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:37.964660 26602 tablet_replica.cc:439] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc: TxnStatusTablet state changed. Reason: New leader b299670f91894c27af07585254dd56f4. Latest consensus state: current_term: 1 leader_uuid: "b299670f91894c27af07585254dd56f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } } }
I20260806 12:56:37.964700 26576 tablet_replica.cc:439] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c: TxnStatusTablet state changed. Reason: New leader b299670f91894c27af07585254dd56f4. Latest consensus state: current_term: 1 leader_uuid: "b299670f91894c27af07585254dd56f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } } }
I20260806 12:56:37.964748 26602 tablet_replica.cc:442] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:37.964775 26576 tablet_replica.cc:442] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:37.965425 26577 tablet_replica.cc:439] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b299670f91894c27af07585254dd56f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } } }
I20260806 12:56:37.965497 26577 tablet_replica.cc:442] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:56:37.965627 26602 tablet_replica.cc:439] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b299670f91894c27af07585254dd56f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } } }
I20260806 12:56:37.965651 26584 mvcc.cc:205] Tried to move back new op lower bound from 7315542007658565632 to 7315542007617769472. Current Snapshot: MvccSnapshot[applied={T|T < 7315542007658565632}]
I20260806 12:56:37.965695 26602 tablet_replica.cc:442] T 1024a99f3d2c4670be838054e0eb1892 P d0f7be3be5d3431294e18ab9bd0363dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:37.966332 26588 tablet_replica.cc:439] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b299670f91894c27af07585254dd56f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } } }
I20260806 12:56:37.966414 26588 tablet_replica.cc:442] T 1024a99f3d2c4670be838054e0eb1892 P b299670f91894c27af07585254dd56f4: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:56:37.966563 26576 tablet_replica.cc:439] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b299670f91894c27af07585254dd56f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f7be3be5d3431294e18ab9bd0363dc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 35563 } } peers { permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 } } peers { permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 } } }
I20260806 12:56:37.966634 26576 tablet_replica.cc:442] T 1024a99f3d2c4670be838054e0eb1892 P fd2215bb98be4f96bb7556740480e13c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:37.970345 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.967876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53750 (local address 127.24.83.1:35563)
0806 12:56:37.967934 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.967936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.967945 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:56:37.967999 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.968000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.968017 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.968096 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:37.968170 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.968885 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.969548 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.969831 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.969898 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.969997 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:37.970132 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:37.970197 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.970257 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.970279 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:56:37.976352 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.972988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:52629 (local address 127.24.83.3:45777)
0806 12:56:37.973037 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.973039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.973145 (+ 106us) server_negotiation.cc:407] Connection header received
0806 12:56:37.973217 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.973218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.973229 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.973295 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:37.973611 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.974093 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.975636 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.975834 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.975923 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.976111 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:37.976150 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:37.976213 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.976265 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.976288 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 12:56:37.976457 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.972917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:45777 (local address 127.24.83.1:52629)
0806 12:56:37.973073 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.973084 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.973160 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.973323 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.973324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.973326 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:56:37.973537 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.973539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.974261 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.974262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.975527 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.975530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.975923 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.975924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.976010 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.976024 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:56:37.976214 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:56:37.976217 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.976263 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.976308 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260806 12:56:37.978889 26504 raft_consensus.cc:1261] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4 [term 1 FOLLOWER]: Refusing update from remote peer d0f7be3be5d3431294e18ab9bd0363dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:37.979205 26360 raft_consensus.cc:1261] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c [term 1 FOLLOWER]: Refusing update from remote peer d0f7be3be5d3431294e18ab9bd0363dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:37.979233 26578 consensus_queue.cc:1035] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b299670f91894c27af07585254dd56f4" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 45777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:37.979413 26602 consensus_queue.cc:1035] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd2215bb98be4f96bb7556740480e13c" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:37.988677 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.985818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:35481 (local address 127.24.83.1:35563)
0806 12:56:37.985852 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:37.985854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:37.986042 (+ 188us) server_negotiation.cc:407] Connection header received
0806 12:56:37.986116 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:37.986117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:37.986127 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:37.986178 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:37.986331 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.986782 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.987932 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.988142 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.988225 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.988428 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:37.988460 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:37.988532 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:37.988592 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 12:56:37.988611 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 12:56:37.988708 26627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:37.985782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:35563 (local address 127.24.83.3:35481)
0806 12:56:37.985969 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:37.985979 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:37.986058 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:37.986182 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:37.986182 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:37.986184 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:56:37.986245 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.986247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.986863 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.986864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.987835 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:37.987837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:37.988227 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:37.988228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:37.988319 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:37.988332 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:56:37.988532 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:56:37.988534 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 12:56:37.988588 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 12:56:37.988631 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":72,"threads_started":1}
I20260806 12:56:37.997752 26057 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:37.997851 26057 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:37.998692 26057 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e7eb8cc13364a489a1945f4d44e508f: Sending DeleteTablet for 3 replicas of tablet 52f758d777454f5a804a246192bb2eb8
I20260806 12:56:37.998972 26340 tablet_service.cc:1514] Processing DeleteTablet for tablet 52f758d777454f5a804a246192bb2eb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:37 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:44684
I20260806 12:56:37.998982 26484 tablet_service.cc:1514] Processing DeleteTablet for tablet 52f758d777454f5a804a246192bb2eb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:37 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56910
I20260806 12:56:37.999090 26594 tablet_replica.cc:330] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4: stopping tablet replica
I20260806 12:56:37.999114 26596 tablet_replica.cc:330] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c: stopping tablet replica
I20260806 12:56:37.999116 26197 tablet_service.cc:1514] Processing DeleteTablet for tablet 52f758d777454f5a804a246192bb2eb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:37 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:53734
I20260806 12:56:37.999143 26594 raft_consensus.cc:2229] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:37.999163 26596 raft_consensus.cc:2229] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:37.999178 26594 raft_consensus.cc:2258] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:37.999188 26595 tablet_replica.cc:330] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc: stopping tablet replica
I20260806 12:56:37.999197 26596 raft_consensus.cc:2258] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:37.999231 26595 raft_consensus.cc:2229] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:56:37.999315 26595 raft_consensus.cc:2258] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:37.999637 26596 ts_tablet_manager.cc:1905] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:37.999692 26594 ts_tablet_manager.cc:1905] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:38.001000 26595 ts_tablet_manager.cc:1905] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:38.001379 26596 ts_tablet_manager.cc:1918] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:56:38.001430 26596 log.cc:1196] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/52f758d777454f5a804a246192bb2eb8
I20260806 12:56:38.001646 26596 ts_tablet_manager.cc:1939] T 52f758d777454f5a804a246192bb2eb8 P fd2215bb98be4f96bb7556740480e13c: Deleting consensus metadata
I20260806 12:56:38.001987 26043 catalog_manager.cc:4841] TS fd2215bb98be4f96bb7556740480e13c (127.24.83.2:44505): tablet 52f758d777454f5a804a246192bb2eb8 (table test-table [id=255ec83dde0c44e584851f65108e8ee0]) successfully deleted
I20260806 12:56:38.002911 26595 ts_tablet_manager.cc:1918] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:56:38.002964 26595 log.cc:1196] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-0/wal/wals/52f758d777454f5a804a246192bb2eb8
I20260806 12:56:38.003198 26595 ts_tablet_manager.cc:1939] T 52f758d777454f5a804a246192bb2eb8 P d0f7be3be5d3431294e18ab9bd0363dc: Deleting consensus metadata
I20260806 12:56:38.003572 26043 catalog_manager.cc:4841] TS d0f7be3be5d3431294e18ab9bd0363dc (127.24.83.1:35563): tablet 52f758d777454f5a804a246192bb2eb8 (table test-table [id=255ec83dde0c44e584851f65108e8ee0]) successfully deleted
I20260806 12:56:38.004971 26594 ts_tablet_manager.cc:1918] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:56:38.005038 26594 log.cc:1196] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/52f758d777454f5a804a246192bb2eb8
I20260806 12:56:38.005301 26594 ts_tablet_manager.cc:1939] T 52f758d777454f5a804a246192bb2eb8 P b299670f91894c27af07585254dd56f4: Deleting consensus metadata
I20260806 12:56:38.005641 26042 catalog_manager.cc:4841] TS b299670f91894c27af07585254dd56f4 (127.24.83.3:45777): tablet 52f758d777454f5a804a246192bb2eb8 (table test-table [id=255ec83dde0c44e584851f65108e8ee0]) successfully deleted
Aug 06 12:56:38 dist-test-slave-zkpd krb5kdc[26011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020997, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:56:38.016693 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:38.004201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53758 (local address 127.24.83.1:35563)
0806 12:56:38.005574 (+ 1373us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:38.005576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:38.005755 (+ 179us) server_negotiation.cc:407] Connection header received
0806 12:56:38.005863 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:38.005865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:38.005875 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:38.005942 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:38.006670 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:38.007429 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:38.008690 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:38.008991 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:38.009056 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:38.013088 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:38.013096 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:38.013097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:38.013104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:38.015231 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:38.015634 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:38.015638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:38.015638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:38.015704 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:38.015952 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:38.015952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:38.015953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:38.016117 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:38.016264 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:38.016503 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 12:56:38.016585 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1348}
W20260806 12:56:38.017321 26257 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53758
I20260806 12:56:38.026491 26610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:38.018340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43604 (local address 127.24.83.62:33351)
0806 12:56:38.018383 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:38.018385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:38.018575 (+ 190us) server_negotiation.cc:407] Connection header received
0806 12:56:38.018626 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:38.018627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:38.018637 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:38.018697 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:38.019443 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:38.020274 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:38.021319 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:38.021642 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:38.021714 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:38.023178 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:38.023179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:38.023179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:38.023187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:38.024989 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:38.025463 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:38.025465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:38.025465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:38.025528 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:38.025842 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:38.025843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:38.025844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:38.025988 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:38.026092 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:38.026295 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 12:56:38.026395 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 12:56:38.026904 26057 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43604
I20260806 12:56:38.027619 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 26129
I20260806 12:56:38.034870 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 26272
I20260806 12:56:38.040964 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 26416
I20260806 12:56:38.048384 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 26024
2026-08-06T12:56:38Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2599 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:56:38 dist-test-slave-zkpd krb5kdc[26639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:56:38 dist-test-slave-zkpd krb5kdc[26639](info): set up 2 sockets
Aug 06 12:56:38 dist-test-slave-zkpd krb5kdc[26639](info): commencing operation
krb5kdc: starting...
W20260806 12:56:39.091737 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:56:39 dist-test-slave-zkpd krb5kdc[26639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:56:39.209041 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:44739
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:44739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:39.290663 26652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:39.290891 26652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:39.290915 26652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:39.292140 26652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:39.292182 26652 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 12:56:39.292196 26652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:39.292209 26652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:39.292222 26652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:39.292233 26652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:39.293910 26652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:44739
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:44739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:39.294175 26652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:39.294420 26652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:39.299365 26658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:39.299345 26657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:39.299333 26660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:39.299705 26652 server_base.cc:1034] running on GCE node
I20260806 12:56:39.300010 26652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:39.300318 26652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:39.301466 26652 hybrid_clock.cc:648] HybridClock initialized: now 1786020999301446 us; error 35 us; skew 500 ppm
Aug 06 12:56:39 dist-test-slave-zkpd krb5kdc[26639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:39.304188 26652 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:56:39.304697 26652 webserver.cc:460] Webserver started at http://127.24.83.62:35399/ using document root <none> and password file <none>
I20260806 12:56:39.304941 26652 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:39.304989 26652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:39.305124 26652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:39.306190 26652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "4e6c29fedee94c65a4f346199331ae41"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "edd3ab6e15a840752d1e946fdbb3e906"
server_key_iv: "5e7499ab008fa673274f27e3df6c0266"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.306568 26652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "4e6c29fedee94c65a4f346199331ae41"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "edd3ab6e15a840752d1e946fdbb3e906"
server_key_iv: "5e7499ab008fa673274f27e3df6c0266"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.308038 26652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:39.308871 26667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:39.309062 26652 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:39.309152 26652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "4e6c29fedee94c65a4f346199331ae41"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "edd3ab6e15a840752d1e946fdbb3e906"
server_key_iv: "5e7499ab008fa673274f27e3df6c0266"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.309212 26652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:39.326116 26652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:39.328421 26652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:39.328529 26652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:39.334087 26652 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:44739
I20260806 12:56:39.334188 26732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:44739 every 8 connection(s)
I20260806 12:56:39.334524 26652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:39.335180 26733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:39.337561 26733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41: Bootstrap starting.
I20260806 12:56:39.338156 26733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:39.338570 26733 log.cc:824] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:39.339247 26733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41: No bootstrap required, opened a new log
I20260806 12:56:39.341506 26733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6c29fedee94c65a4f346199331ae41" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44739 } }
I20260806 12:56:39.341670 26733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:39.341714 26733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e6c29fedee94c65a4f346199331ae41, State: Initialized, Role: FOLLOWER
I20260806 12:56:39.341902 26733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6c29fedee94c65a4f346199331ae41" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44739 } }
I20260806 12:56:39.342003 26733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:39.342049 26733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:39.342098 26733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:39.342629 26733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6c29fedee94c65a4f346199331ae41" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44739 } }
I20260806 12:56:39.342747 26733 leader_election.cc:302] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e6c29fedee94c65a4f346199331ae41; no voters:
I20260806 12:56:39.342929 26733 leader_election.cc:288] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:39.342978 26736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:39.343173 26736 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [term 1 LEADER]: Becoming Leader. State: Replica: 4e6c29fedee94c65a4f346199331ae41, State: Running, Role: LEADER
I20260806 12:56:39.343185 26733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:39.343274 26736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6c29fedee94c65a4f346199331ae41" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44739 } }
I20260806 12:56:39.343641 26737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e6c29fedee94c65a4f346199331ae41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6c29fedee94c65a4f346199331ae41" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44739 } } }
I20260806 12:56:39.343645 26738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e6c29fedee94c65a4f346199331ae41. Latest consensus state: current_term: 1 leader_uuid: "4e6c29fedee94c65a4f346199331ae41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6c29fedee94c65a4f346199331ae41" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44739 } } }
I20260806 12:56:39.343766 26737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:39.343775 26738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:39.344085 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 26652
I20260806 12:56:39.344185 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:39.344112 26745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:39.344362 24908 external_mini_cluster.cc:1428] Setting key c7f981443f826a5f0734be45f199c32c
I20260806 12:56:39.344805 26745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:39.348333 26745 catalog_manager.cc:1347] Generated new cluster ID: 4f7d8393d2b44dbeba3005839ffc5add
I20260806 12:56:39.348387 26745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:56:39 dist-test-slave-zkpd krb5kdc[26639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:39.356284 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.344904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44144 (local address 127.24.83.62:44739)
0806 12:56:39.345180 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:39.345190 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:39.345220 (+ 30us) server_negotiation.cc:407] Connection header received
0806 12:56:39.345514 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:39.345522 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:39.345583 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:39.346514 (+ 931us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:39.346608 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.348014 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.349114 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.349742 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.349869 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.352638 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:39.352642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:39.352642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:39.352654 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:39.354580 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.355004 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.355007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.355008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.355133 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.355361 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.355362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.355362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.355635 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:39.355768 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:39.355928 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 12:56:39.356053 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":83,"threads_started":1}
I20260806 12:56:39.367865 26745 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:39.368409 26745 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:39.384166 26745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41: Generated new TSK 0
I20260806 12:56:39.384356 26745 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:56:39.416929 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:44739
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:39.501045 26759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:39.501247 26759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:39.501271 26759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:39.502396 26759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:39.502436 26759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:39.502480 26759 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 12:56:39.502507 26759 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:56:39.502552 26759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:56:39.504045 26759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.83.62:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:39.504280 26759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:39.504506 26759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:39.509521 26764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:39.509498 26765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:39.509694 26759 server_base.cc:1034] running on GCE node
W20260806 12:56:39.509779 26767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:39.509984 26759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:39.510212 26759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:39.511368 26759 hybrid_clock.cc:648] HybridClock initialized: now 1786020999511353 us; error 32 us; skew 500 ppm
Aug 06 12:56:39 dist-test-slave-zkpd krb5kdc[26639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:39.514060 26759 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:56:39.514444 26759 webserver.cc:460] Webserver started at http://127.24.83.1:37731/ using document root <none> and password file <none>
I20260806 12:56:39.514619 26759 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:39.514659 26759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:39.514746 26759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:39.515717 26759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "964bbf2a9f984c71820f3b340debf1a6"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "11bc54cb385fa974475f98407d15fcae"
server_key_iv: "c5138d3c13084a181c73d997962303e8"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.516094 26759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "964bbf2a9f984c71820f3b340debf1a6"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "11bc54cb385fa974475f98407d15fcae"
server_key_iv: "c5138d3c13084a181c73d997962303e8"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.517351 26759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:56:39.518014 26774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:39.518203 26759 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:56:39.518282 26759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "964bbf2a9f984c71820f3b340debf1a6"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "11bc54cb385fa974475f98407d15fcae"
server_key_iv: "c5138d3c13084a181c73d997962303e8"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.518332 26759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:39.534881 26759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:39.536654 26759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:39.536761 26759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:39.537230 26759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:39.537271 26759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:39.537297 26759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:39.537348 26759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:39.544162 26759 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:42947
Aug 06 12:56:39 dist-test-slave-zkpd krb5kdc[26639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:39.544548 26759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:56:39.544643 26891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:42947 every 8 connection(s)
I20260806 12:56:39.547708 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.537754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:53533 (local address 127.24.83.62:44739)
0806 12:56:39.537841 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:39.537844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:39.537998 (+ 154us) server_negotiation.cc:407] Connection header received
0806 12:56:39.538472 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:39.538474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:39.538492 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:39.538572 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:39.539450 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.540324 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.541845 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.542210 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.542296 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.545114 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:39.545117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:39.545117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:39.545126 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:39.546613 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.546967 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.546968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.546968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.547017 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.547184 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.547185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.547185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.547281 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:39.547341 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:39.547575 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 12:56:39.547644 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 12:56:39.547860 26784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.537674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:44739 (local address 127.24.83.1:53533)
0806 12:56:39.537867 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:39.537908 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:39.538375 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:39.538666 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:39.538671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:39.539032 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:39.539332 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.539340 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.540431 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.540432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.541755 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.541759 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.542302 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.542302 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.542397 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.542959 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:39.542969 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:39.544962 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:39.546684 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.546688 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.546689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.546900 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.547056 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.547056 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.547057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.547134 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.547350 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:39.547355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:39.547466 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 12:56:39.547546 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 12:56:39.547657 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260806 12:56:39.548360 26892 heartbeater.cc:347] Connected to a master server at 127.24.83.62:44739
I20260806 12:56:39.548463 26892 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:39.548630 26892 heartbeater.cc:511] Master 127.24.83.62:44739 requested a full tablet report, sending...
I20260806 12:56:39.549185 26683 ts_manager.cc:194] Registered new tserver with Master: 964bbf2a9f984c71820f3b340debf1a6 (127.24.83.1:42947)
I20260806 12:56:39.549901 26683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:53533
I20260806 12:56:39.551577 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 26759
I20260806 12:56:39.551685 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:56:39.551857 24908 external_mini_cluster.cc:1428] Setting key 3b967ee11275835e6d75b26a573fd684
I20260806 12:56:39.554962 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.549430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:48503 (local address 127.24.83.62:44739)
0806 12:56:39.549466 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:39.549468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:39.549605 (+ 137us) server_negotiation.cc:407] Connection header received
0806 12:56:39.549676 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:39.549676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:39.549687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:39.549731 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:39.550052 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.550593 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.551286 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.551514 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.551599 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.552505 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:39.552506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:39.552507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:39.552513 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:39.553849 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.554201 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.554202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.554202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.554256 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.554473 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.554474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.554474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.554581 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:39.554655 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:39.554805 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 12:56:39.554876 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 12:56:39.554975 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.549399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:44739 (local address 127.24.83.1:48503)
0806 12:56:39.549528 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:39.549541 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:39.549618 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:39.549743 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:39.549744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:39.549917 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:39.549988 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.549990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.550696 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.550697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.551200 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.551202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.551601 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.551602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.551678 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.552143 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:39.552147 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:39.552400 (+ 253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:39.553946 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.553947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.553948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.554116 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.554332 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.554333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.554333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.554394 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.554649 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:39.554649 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:39.554694 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 12:56:39.554781 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 12:56:39.554878 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:56:39.576965 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:44739
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:39.657830 26902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:39.658022 26902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:39.658047 26902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:39.659180 26902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:39.659224 26902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:39.659245 26902 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 12:56:39.659303 26902 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:56:39.659332 26902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:56:39.660779 26902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.83.62:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:39.660959 26902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:39.661202 26902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:39.666153 26907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:39.666145 26910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:39.666396 26902 server_base.cc:1034] running on GCE node
W20260806 12:56:39.666145 26908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:39.666618 26902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:39.666916 26902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:39.668079 26902 hybrid_clock.cc:648] HybridClock initialized: now 1786020999668058 us; error 38 us; skew 500 ppm
Aug 06 12:56:39 dist-test-slave-zkpd krb5kdc[26639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:39.670737 26902 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:56:39.671177 26902 webserver.cc:460] Webserver started at http://127.24.83.2:42157/ using document root <none> and password file <none>
I20260806 12:56:39.671411 26902 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:39.671479 26902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:39.671629 26902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:39.672704 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "505a820efa3e45b484a44a0daf95f373"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "c020bc968497ed6e2f664d1b0f26c4fc"
server_key_iv: "5b007369f742c3354350dd34fddbd015"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.673061 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "505a820efa3e45b484a44a0daf95f373"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "c020bc968497ed6e2f664d1b0f26c4fc"
server_key_iv: "5b007369f742c3354350dd34fddbd015"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.674368 26902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 12:56:39.675056 26918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:39.675225 26902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:56:39.675336 26902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "505a820efa3e45b484a44a0daf95f373"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "c020bc968497ed6e2f664d1b0f26c4fc"
server_key_iv: "5b007369f742c3354350dd34fddbd015"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.675416 26902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:39.699327 26902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:39.701858 26902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:39.702021 26902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:39.702553 26902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:39.702616 26902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:39.702662 26902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:39.702708 26902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:56:39 dist-test-slave-zkpd krb5kdc[26639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:39.710142 26902 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:37801
I20260806 12:56:39.710227 27035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:37801 every 8 connection(s)
I20260806 12:56:39.710556 26902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:56:39.711817 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 26902
I20260806 12:56:39.711944 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:56:39.712143 24908 external_mini_cluster.cc:1428] Setting key ea0a96bcaebdc744054c6731250ceed6
I20260806 12:56:39.713330 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.703136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:59325 (local address 127.24.83.62:44739)
0806 12:56:39.703182 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:39.703185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:39.703419 (+ 234us) server_negotiation.cc:407] Connection header received
0806 12:56:39.704035 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:39.704037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:39.704051 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:39.704114 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:39.705041 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.705854 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.707587 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.707820 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.707941 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.710138 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:39.710140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:39.710140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:39.710149 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:39.711759 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.712279 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.712280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.712281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.712365 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.712614 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.712615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.712615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.712748 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:39.712833 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:39.713149 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 12:56:39.713237 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 12:56:39.713583 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.703124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:44739 (local address 127.24.83.2:59325)
0806 12:56:39.703285 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:39.703321 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:39.703848 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:39.704194 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:39.704198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:39.704583 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:39.704919 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.704928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.705994 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.705995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.707462 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.707466 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.707945 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.707946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.708084 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.708604 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:39.708612 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:39.710042 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:39.711865 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.711870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.711886 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.712209 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.712451 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.712452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.712452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.712558 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.712839 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:39.712845 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:39.712989 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 12:56:39.713092 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 12:56:39.713400 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260806 12:56:39.714169 27036 heartbeater.cc:347] Connected to a master server at 127.24.83.62:44739
I20260806 12:56:39.714272 27036 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:39.714470 27036 heartbeater.cc:511] Master 127.24.83.62:44739 requested a full tablet report, sending...
I20260806 12:56:39.714824 26683 ts_manager.cc:194] Registered new tserver with Master: 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801)
I20260806 12:56:39.715350 26683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:59325
I20260806 12:56:39.721485 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.715172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:40113 (local address 127.24.83.62:44739)
0806 12:56:39.715215 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:39.715216 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:39.715343 (+ 127us) server_negotiation.cc:407] Connection header received
0806 12:56:39.715415 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:39.715416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:39.715426 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:39.715473 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:39.715873 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.716428 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.717526 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.717753 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.717803 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.718850 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:39.718851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:39.718851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:39.718857 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:39.720301 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.720719 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.720720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.720720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.720813 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.721003 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.721004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.721004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.721132 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:39.721180 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:39.721367 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 12:56:39.721425 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 12:56:39.721613 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.715127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:44739 (local address 127.24.83.2:40113)
0806 12:56:39.715266 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:39.715275 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:39.715356 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:39.715517 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:39.715518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:39.715723 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:39.715789 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.715791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.716534 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.716535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.717418 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.717421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.717832 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.717833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.717916 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.718319 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:39.718325 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:39.718754 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:39.720382 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.720383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.720384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.720609 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.720864 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.720865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.720865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.720939 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.721200 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:39.721201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:39.721255 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 12:56:39.721352 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 12:56:39.721462 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:56:39.738664 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:44739
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:39.820676 27045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:39.820936 27045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:39.820969 27045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:39.822202 27045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:39.822242 27045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:39.822263 27045 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 12:56:39.822275 27045 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:56:39.822293 27045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:56:39.823671 27045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.83.62:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:39.823865 27045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:39.824100 27045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:56:39.829209 27045 server_base.cc:1034] running on GCE node
W20260806 12:56:39.829183 27050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:39.829169 27051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:39.829173 27053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:39.829581 27045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:39.829852 27045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:39.831015 27045 hybrid_clock.cc:648] HybridClock initialized: now 1786020999830999 us; error 46 us; skew 500 ppm
Aug 06 12:56:39 dist-test-slave-zkpd krb5kdc[26639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:39.833477 27045 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:56:39.833938 27045 webserver.cc:460] Webserver started at http://127.24.83.3:33487/ using document root <none> and password file <none>
I20260806 12:56:39.834133 27045 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:39.834174 27045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:39.834262 27045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:39.835216 27045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "1e7a022b41c8447abb35f2782a0ea165"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "fc0f6f8e80b52625b6d0a71654455091"
server_key_iv: "3cc9f7a3bdb5ccaf6258f9021a8faee6"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.835517 27045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "1e7a022b41c8447abb35f2782a0ea165"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "fc0f6f8e80b52625b6d0a71654455091"
server_key_iv: "3cc9f7a3bdb5ccaf6258f9021a8faee6"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.836920 27045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:56:39.837607 27060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:39.837767 27045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:56:39.837854 27045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "1e7a022b41c8447abb35f2782a0ea165"
format_stamp: "Formatted at 2026-08-06 12:56:39 on dist-test-slave-zkpd"
server_key: "fc0f6f8e80b52625b6d0a71654455091"
server_key_iv: "3cc9f7a3bdb5ccaf6258f9021a8faee6"
server_key_version: "encryptionkey@0"
I20260806 12:56:39.837916 27045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:39.853295 27045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:39.855123 27045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:39.855235 27045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:39.855676 27045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:39.855716 27045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:39.855741 27045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:39.855795 27045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:56:39 dist-test-slave-zkpd krb5kdc[26639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:39.863221 27045 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:46467
I20260806 12:56:39.863281 27179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:46467 every 8 connection(s)
I20260806 12:56:39.863664 27045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:56:39.866191 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.856228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:55629 (local address 127.24.83.62:44739)
0806 12:56:39.856511 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:39.856513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:39.856527 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:56:39.857046 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:39.857047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:39.857070 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:39.857156 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:39.857818 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.858661 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.860330 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.860680 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.860772 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.863687 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:39.863689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:39.863689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:39.863696 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:39.865079 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.865449 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.865450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.865451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.865499 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.865672 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.865673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.865673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.865772 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:39.865831 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:39.866064 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 12:56:39.866128 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260806 12:56:39.866365 27071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.856200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:44739 (local address 127.24.83.3:55629)
0806 12:56:39.856394 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:39.856448 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:39.856938 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:39.857238 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:39.857242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:39.857476 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:39.857711 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.857716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.858762 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.858764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.860230 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.860233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.860808 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.860809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.860955 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.861568 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:39.861583 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:39.863576 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:39.865152 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.865157 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.865159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.865385 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.865539 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.865539 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.865540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.865622 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.865839 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:39.865844 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:39.865952 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 12:56:39.866035 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 12:56:39.866152 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":92,"threads_started":1}
I20260806 12:56:39.866875 27180 heartbeater.cc:347] Connected to a master server at 127.24.83.62:44739
I20260806 12:56:39.866957 27180 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:39.867164 27180 heartbeater.cc:511] Master 127.24.83.62:44739 requested a full tablet report, sending...
I20260806 12:56:39.867565 26683 ts_manager.cc:194] Registered new tserver with Master: 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467)
I20260806 12:56:39.868116 26683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:55629
I20260806 12:56:39.872969 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.867893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:44739 (local address 127.24.83.3:45225)
0806 12:56:39.868059 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:39.868071 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:39.868134 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:39.868317 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:39.868318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:39.868534 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:39.868594 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.868596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.869200 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.869201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.869668 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.869670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.869971 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.869971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.870023 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.870449 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:39.870453 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:39.870749 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:39.872154 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.872155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.872155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.872321 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.872486 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:39.872487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:39.872487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:39.872536 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:39.872750 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:39.872751 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:39.872784 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 12:56:39.872836 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 12:56:39.872908 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":78,"threads_started":1}
I20260806 12:56:39.872974 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.867998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:45225 (local address 127.24.83.62:44739)
0806 12:56:39.868043 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:39.868045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:39.868141 (+ 96us) server_negotiation.cc:407] Connection header received
0806 12:56:39.868204 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:39.868205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:39.868216 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:39.868273 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:39.868674 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.869109 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.869733 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.869907 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.869958 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.870811 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:39.870812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:39.870812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:39.870816 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:39.872086 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.872382 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.872382 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.872383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.872430 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.872585 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.872586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.872586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.872673 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:39.872720 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:39.872866 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 12:56:39.872921 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:56:39.873360 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 27045
I20260806 12:56:39.873440 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:56:39.873628 24908 external_mini_cluster.cc:1428] Setting key d62545a4aa9f0c0f9cfa8d3c7e6f7abb
I20260806 12:56:39.874639 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 12:56:39 dist-test-slave-zkpd krb5kdc[26639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 12:56:39 dist-test-slave-zkpd krb5kdc[26639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:39.897075 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.890636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44152 (local address 127.24.83.62:44739)
0806 12:56:39.890695 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:39.890697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:39.890705 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:56:39.890729 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:39.890729 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:39.890738 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:39.890792 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:39.891307 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.891776 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.892516 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.892692 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.892745 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.894880 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:39.894881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:39.894882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:39.894887 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:39.896174 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.896502 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.896503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.896503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.896552 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:39.896722 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:39.896723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:39.896723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:39.896813 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:39.896883 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:39.896961 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 12:56:39.897012 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 12:56:39.898835 26683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:56:39.900514 26683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:56:39.907109 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.902985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:46467 (local address 127.0.0.1:39634)
0806 12:56:39.903399 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:39.903410 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:39.903465 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:39.903714 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:39.903714 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:39.903716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:39.903757 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.903759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.905019 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.905020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.905993 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.905996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.906761 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.906762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.906839 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.906854 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 12:56:39.906955 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 12:56:39.907008 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260806 12:56:39.907107 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.903093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39634 (local address 127.24.83.3:46467)
0806 12:56:39.903394 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:39.903399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:39.903526 (+ 127us) server_negotiation.cc:407] Connection header received
0806 12:56:39.903569 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:39.903571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:39.903587 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:39.903654 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:39.903830 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.904933 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.906104 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.906656 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.906756 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.906817 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:39.906972 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 12:56:39.907009 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":83,"threads_started":1}
I20260806 12:56:39.907414 27197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.902769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:37801 (local address 127.0.0.1:40216)
0806 12:56:39.902940 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:39.902980 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:39.903043 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:39.903300 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:39.903303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:39.903305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:39.903372 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.903373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.904825 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.904826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.906132 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.906135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.907140 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.907141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.907215 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.907219 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:56:39.907311 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 12:56:39.907347 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":59,"thread_start_us":67,"threads_started":1}
I20260806 12:56:39.907549 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.902904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.24.83.2:37801)
0806 12:56:39.903121 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:39.903127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:39.903138 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:56:39.903180 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:39.903182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:39.903197 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:39.903297 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:39.903518 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.904742 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.906241 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.906981 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.907031 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.907083 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:39.907335 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 12:56:39.907383 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":96,"threads_started":1}
I20260806 12:56:39.907843 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.902769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:42947 (local address 127.0.0.1:56814)
0806 12:56:39.903150 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:39.903159 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:39.903196 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:39.903669 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:39.903670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:39.903672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:39.903715 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.903717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.904681 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.904682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.906132 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:39.906155 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.907136 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.907138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:39.907664 (+ 526us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.907669 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:56:39.907725 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 12:56:39.907760 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":331,"thread_start_us":69,"threads_started":1}
I20260806 12:56:39.908088 27203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:39.903165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.24.83.1:42947)
0806 12:56:39.903353 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:39.903357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:39.903369 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:56:39.903415 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:39.903416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:39.903434 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:39.903598 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:39.903795 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.904590 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.906240 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:39.906981 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:39.907050 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:39.907098 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:39.907946 (+ 848us) server_negotiation.cc:299] Negotiation successful
0806 12:56:39.908003 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260806 12:56:39.907812 27113 tablet_service.cc:1467] Processing CreateTablet for tablet ed95f9454b9a4038818a21dea4834e1c (DEFAULT_TABLE table=test-table [id=7b83261499064fde99eeb265f8c34289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:39.908196 26969 tablet_service.cc:1467] Processing CreateTablet for tablet ed95f9454b9a4038818a21dea4834e1c (DEFAULT_TABLE table=test-table [id=7b83261499064fde99eeb265f8c34289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:39.908561 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed95f9454b9a4038818a21dea4834e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:39.908568 26969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed95f9454b9a4038818a21dea4834e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:39.908802 26826 tablet_service.cc:1467] Processing CreateTablet for tablet ed95f9454b9a4038818a21dea4834e1c (DEFAULT_TABLE table=test-table [id=7b83261499064fde99eeb265f8c34289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:39.909111 26826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed95f9454b9a4038818a21dea4834e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:39.911276 27204 tablet_bootstrap.cc:492] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165: Bootstrap starting.
I20260806 12:56:39.912257 27204 tablet_bootstrap.cc:654] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:39.912421 27205 tablet_bootstrap.cc:492] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: Bootstrap starting.
I20260806 12:56:39.912739 27204 log.cc:824] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:39.912801 27206 tablet_bootstrap.cc:492] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: Bootstrap starting.
I20260806 12:56:39.913154 27205 tablet_bootstrap.cc:654] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:39.913465 27206 tablet_bootstrap.cc:654] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:39.913580 27205 log.cc:824] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:39.913837 27206 log.cc:824] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:39.913954 27204 tablet_bootstrap.cc:492] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165: No bootstrap required, opened a new log
I20260806 12:56:39.914027 27204 ts_tablet_manager.cc:1397] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 12:56:39.914346 27205 tablet_bootstrap.cc:492] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: No bootstrap required, opened a new log
I20260806 12:56:39.914412 27205 ts_tablet_manager.cc:1397] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:56:39.914604 27206 tablet_bootstrap.cc:492] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: No bootstrap required, opened a new log
I20260806 12:56:39.914669 27206 ts_tablet_manager.cc:1397] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:56:39.915797 27204 raft_consensus.cc:348] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } }
I20260806 12:56:39.916003 27204 raft_consensus.cc:374] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:39.916049 27204 raft_consensus.cc:729] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7a022b41c8447abb35f2782a0ea165, State: Initialized, Role: FOLLOWER
I20260806 12:56:39.916155 27204 consensus_queue.cc:260] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } }
I20260806 12:56:39.916464 27180 heartbeater.cc:503] Master 127.24.83.62:44739 was elected leader, sending a full tablet report...
I20260806 12:56:39.916447 27204 ts_tablet_manager.cc:1428] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:56:39.916469 27205 raft_consensus.cc:348] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } }
I20260806 12:56:39.916631 27205 raft_consensus.cc:374] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:39.916671 27205 raft_consensus.cc:729] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 505a820efa3e45b484a44a0daf95f373, State: Initialized, Role: FOLLOWER
I20260806 12:56:39.916651 27206 raft_consensus.cc:348] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } }
I20260806 12:56:39.916785 27206 raft_consensus.cc:374] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:39.916817 27206 raft_consensus.cc:729] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964bbf2a9f984c71820f3b340debf1a6, State: Initialized, Role: FOLLOWER
I20260806 12:56:39.916772 27205 consensus_queue.cc:260] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } }
I20260806 12:56:39.916929 27206 consensus_queue.cc:260] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } }
I20260806 12:56:39.917052 27205 ts_tablet_manager.cc:1428] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 12:56:39.917109 27036 heartbeater.cc:503] Master 127.24.83.62:44739 was elected leader, sending a full tablet report...
I20260806 12:56:39.917179 27206 ts_tablet_manager.cc:1428] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 12:56:39.917303 26892 heartbeater.cc:503] Master 127.24.83.62:44739 was elected leader, sending a full tablet report...
W20260806 12:56:39.961485 27037 tablet.cc:2369] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:56:40.011075 27213 raft_consensus.cc:482] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:56:40.011209 27213 raft_consensus.cc:504] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } }
I20260806 12:56:40.011621 27213 leader_election.cc:288] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801), 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467)
I20260806 12:56:40.015129 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:40.012240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:46949 (local address 127.24.83.3:46467)
0806 12:56:40.012291 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:40.012293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:40.012393 (+ 100us) server_negotiation.cc:407] Connection header received
0806 12:56:40.012470 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:40.012471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:40.012483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:40.012544 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:40.012728 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.013238 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.014420 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.014758 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.014814 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:40.014822 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:40.015004 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 12:56:40.015045 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 12:56:40.015132 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:40.012126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:46467 (local address 127.24.83.1:46949)
0806 12:56:40.012329 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:40.012336 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:40.012407 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:40.012548 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:40.012548 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:40.012550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:40.012650 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:40.012652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.013337 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.013340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:40.014311 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:40.014319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.014833 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.014834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:40.014913 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:40.014918 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:56:40.014972 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 12:56:40.015013 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":132,"threads_started":1}
I20260806 12:56:40.015241 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:40.012229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:37767 (local address 127.24.83.2:37801)
0806 12:56:40.012287 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:40.012291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:40.012301 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:56:40.012335 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:40.012336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:40.012346 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:40.012439 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:40.012670 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.013227 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.014374 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.014862 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.014931 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:40.014941 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:40.015126 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 12:56:40.015171 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:56:40.015249 26784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:40.011994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:37801 (local address 127.24.83.1:37767)
0806 12:56:40.012058 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:40.012071 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:40.012117 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:40.012499 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:40.012500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:40.012502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:40.012582 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:40.012584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.013306 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.013307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:40.014299 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:40.014302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.014945 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.014962 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:40.015045 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:40.015048 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:56:40.015106 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 12:56:40.015143 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 12:56:40.015769 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed95f9454b9a4038818a21dea4834e1c" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505a820efa3e45b484a44a0daf95f373" is_pre_election: true
I20260806 12:56:40.015761 27133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed95f9454b9a4038818a21dea4834e1c" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7a022b41c8447abb35f2782a0ea165" is_pre_election: true
I20260806 12:56:40.015935 26990 raft_consensus.cc:2454] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 0.
I20260806 12:56:40.015941 27133 raft_consensus.cc:2454] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 0.
I20260806 12:56:40.016170 26777 leader_election.cc:302] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 505a820efa3e45b484a44a0daf95f373, 964bbf2a9f984c71820f3b340debf1a6; no voters:
I20260806 12:56:40.016304 27213 raft_consensus.cc:2781] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:56:40.016363 27213 raft_consensus.cc:482] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:56:40.016407 27213 raft_consensus.cc:3037] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:40.017117 27213 raft_consensus.cc:504] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } }
I20260806 12:56:40.017246 27213 leader_election.cc:288] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 election: Requested vote from peers 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801), 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467)
I20260806 12:56:40.017438 27133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed95f9454b9a4038818a21dea4834e1c" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7a022b41c8447abb35f2782a0ea165"
I20260806 12:56:40.017467 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed95f9454b9a4038818a21dea4834e1c" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505a820efa3e45b484a44a0daf95f373"
I20260806 12:56:40.017525 27133 raft_consensus.cc:3037] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:40.017536 26990 raft_consensus.cc:3037] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:40.018294 27133 raft_consensus.cc:2454] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 1.
I20260806 12:56:40.018390 26990 raft_consensus.cc:2454] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 1.
I20260806 12:56:40.018458 26777 leader_election.cc:302] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7a022b41c8447abb35f2782a0ea165, 964bbf2a9f984c71820f3b340debf1a6; no voters:
I20260806 12:56:40.018544 27213 raft_consensus.cc:2781] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:40.018674 27213 raft_consensus.cc:686] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 1 LEADER]: Becoming Leader. State: Replica: 964bbf2a9f984c71820f3b340debf1a6, State: Running, Role: LEADER
I20260806 12:56:40.018781 27213 consensus_queue.cc:237] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } }
I20260806 12:56:40.019608 26684 catalog_manager.cc:5495] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 964bbf2a9f984c71820f3b340debf1a6 (127.24.83.1). New cstate: current_term: 1 leader_uuid: "964bbf2a9f984c71820f3b340debf1a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } health_report { overall_health: HEALTHY } } }
W20260806 12:56:40.045334 26893 tablet.cc:2369] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 12:56:40 dist-test-slave-zkpd krb5kdc[26639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786020999, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:40.083717 27223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:40.076545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:44739 (local address 127.0.0.1:44154)
0806 12:56:40.076747 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:40.076756 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:40.076853 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:40.077029 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:40.077030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:40.077340 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:40.077401 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:40.077403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.078092 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.078094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:40.078601 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:40.078603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.078991 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.078992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:40.079050 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:40.079601 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:40.079606 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:40.080974 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:40.082613 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:40.082621 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:40.082624 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:40.082817 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:40.083028 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:40.083029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:40.083029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:40.083092 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:40.083327 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:40.083335 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:40.083378 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 12:56:40.083469 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 12:56:40.083582 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":110,"threads_started":1}
I20260806 12:56:40.083725 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:40.076608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44154 (local address 127.24.83.62:44739)
0806 12:56:40.076708 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:40.076710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:40.076832 (+ 122us) server_negotiation.cc:407] Connection header received
0806 12:56:40.076935 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:40.076936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:40.076947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:40.077009 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:40.077487 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.078007 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.078678 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.078905 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.078991 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:40.081057 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:40.081058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:40.081059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:40.081065 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:40.082515 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:40.082906 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:40.082907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:40.082908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:40.082962 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:40.083153 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:40.083154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:40.083154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:40.083261 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:40.083336 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:40.083491 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 12:56:40.083645 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":65}
I20260806 12:56:40.085953 26684 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:44154:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 12:56:40.086169 26684 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:56:40.087742 26826 tablet_service.cc:1467] Processing CreateTablet for tablet 69cafa0914ea4623bdd28f61fd9b5253 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=36afed2396b740cd8a11e952800db955]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:56:40.087842 27113 tablet_service.cc:1467] Processing CreateTablet for tablet 69cafa0914ea4623bdd28f61fd9b5253 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=36afed2396b740cd8a11e952800db955]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:56:40.087839 26969 tablet_service.cc:1467] Processing CreateTablet for tablet 69cafa0914ea4623bdd28f61fd9b5253 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=36afed2396b740cd8a11e952800db955]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:56:40.087978 26826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69cafa0914ea4623bdd28f61fd9b5253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:40.088006 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69cafa0914ea4623bdd28f61fd9b5253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:40.088050 26969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69cafa0914ea4623bdd28f61fd9b5253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:40.089362 27205 tablet_bootstrap.cc:492] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373: Bootstrap starting.
I20260806 12:56:40.089505 27204 tablet_bootstrap.cc:492] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: Bootstrap starting.
I20260806 12:56:40.089995 27205 tablet_bootstrap.cc:654] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:40.090011 27204 tablet_bootstrap.cc:654] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:40.090337 27206 tablet_bootstrap.cc:492] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: Bootstrap starting.
I20260806 12:56:40.090703 27204 tablet_bootstrap.cc:492] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: No bootstrap required, opened a new log
I20260806 12:56:40.090772 27206 tablet_bootstrap.cc:654] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:40.090763 27204 ts_tablet_manager.cc:1397] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:56:40.090884 27205 tablet_bootstrap.cc:492] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373: No bootstrap required, opened a new log
I20260806 12:56:40.090934 27205 ts_tablet_manager.cc:1397] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 12:56:40.090953 27204 raft_consensus.cc:348] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.091054 27204 raft_consensus.cc:374] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:40.091080 27204 raft_consensus.cc:729] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7a022b41c8447abb35f2782a0ea165, State: Initialized, Role: FOLLOWER
I20260806 12:56:40.091090 27205 raft_consensus.cc:348] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.091130 27204 consensus_queue.cc:260] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.091162 27205 raft_consensus.cc:374] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:40.091194 27205 raft_consensus.cc:729] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 505a820efa3e45b484a44a0daf95f373, State: Initialized, Role: FOLLOWER
I20260806 12:56:40.091223 27204 ts_tablet_manager.cc:1428] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:40.091233 27205 consensus_queue.cc:260] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.091342 27205 ts_tablet_manager.cc:1428] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 12:56:40.091389 27211 tablet_replica.cc:439] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } }
I20260806 12:56:40.091476 27211 tablet_replica.cc:442] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:40.091468 27212 tablet_replica.cc:439] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } }
I20260806 12:56:40.091557 27212 tablet_replica.cc:442] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:40.091801 27206 tablet_bootstrap.cc:492] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: No bootstrap required, opened a new log
I20260806 12:56:40.091858 27206 ts_tablet_manager.cc:1397] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:56:40.092043 27206 raft_consensus.cc:348] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.092123 27206 raft_consensus.cc:374] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:40.092144 27206 raft_consensus.cc:729] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964bbf2a9f984c71820f3b340debf1a6, State: Initialized, Role: FOLLOWER
I20260806 12:56:40.092195 27206 consensus_queue.cc:260] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.092293 27206 ts_tablet_manager.cc:1428] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:40.092338 27213 tablet_replica.cc:439] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } }
I20260806 12:56:40.092417 27213 tablet_replica.cc:442] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 12:56:40.114564 27181 tablet.cc:2369] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:56:40.158704 27213 raft_consensus.cc:482] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:56:40.158838 27213 raft_consensus.cc:504] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.159037 27213 leader_election.cc:288] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801), 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467)
I20260806 12:56:40.159338 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69cafa0914ea4623bdd28f61fd9b5253" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505a820efa3e45b484a44a0daf95f373" is_pre_election: true
I20260806 12:56:40.159363 27133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69cafa0914ea4623bdd28f61fd9b5253" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7a022b41c8447abb35f2782a0ea165" is_pre_election: true
I20260806 12:56:40.159462 27133 raft_consensus.cc:2454] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 0.
I20260806 12:56:40.159467 26990 raft_consensus.cc:2454] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 0.
I20260806 12:56:40.159654 26777 leader_election.cc:302] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7a022b41c8447abb35f2782a0ea165, 964bbf2a9f984c71820f3b340debf1a6; no voters:
I20260806 12:56:40.159756 27213 raft_consensus.cc:2781] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:56:40.159798 27213 raft_consensus.cc:482] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:56:40.159827 27213 raft_consensus.cc:3037] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:40.160517 27213 raft_consensus.cc:504] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.160656 27213 leader_election.cc:288] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 election: Requested vote from peers 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801), 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467)
I20260806 12:56:40.160862 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69cafa0914ea4623bdd28f61fd9b5253" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505a820efa3e45b484a44a0daf95f373"
I20260806 12:56:40.160907 27133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69cafa0914ea4623bdd28f61fd9b5253" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7a022b41c8447abb35f2782a0ea165"
I20260806 12:56:40.160984 26990 raft_consensus.cc:3037] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:40.160984 27133 raft_consensus.cc:3037] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:40.161801 27133 raft_consensus.cc:2454] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 1.
I20260806 12:56:40.161885 26990 raft_consensus.cc:2454] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 1.
I20260806 12:56:40.162014 26777 leader_election.cc:302] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7a022b41c8447abb35f2782a0ea165, 964bbf2a9f984c71820f3b340debf1a6; no voters:
I20260806 12:56:40.162124 27213 raft_consensus.cc:2781] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:40.162199 27213 raft_consensus.cc:686] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [term 1 LEADER]: Becoming Leader. State: Replica: 964bbf2a9f984c71820f3b340debf1a6, State: Running, Role: LEADER
I20260806 12:56:40.162266 27213 consensus_queue.cc:237] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.162629 27215 tablet_replica.cc:439] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: TxnStatusTablet state changed. Reason: New leader 964bbf2a9f984c71820f3b340debf1a6. Latest consensus state: current_term: 1 leader_uuid: "964bbf2a9f984c71820f3b340debf1a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } }
I20260806 12:56:40.162738 27215 tablet_replica.cc:442] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:56:40.163017 27228 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 12:56:40.162949 26682 catalog_manager.cc:5495] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 964bbf2a9f984c71820f3b340debf1a6 (127.24.83.1). New cstate: current_term: 1 leader_uuid: "964bbf2a9f984c71820f3b340debf1a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } health_report { overall_health: UNKNOWN } } }
I20260806 12:56:40.163146 27228 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 12:56:40.185420 27223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:40.182695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:42947 (local address 127.0.0.1:56820)
0806 12:56:40.182775 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:40.182801 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:40.182856 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:40.183073 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:40.183074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:40.183075 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:56:40.183152 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:40.183154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.183816 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.183818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:40.184459 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:40.184461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.184857 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.184859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:40.184943 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:40.184961 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:56:40.185247 (+ 286us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:56:40.185258 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 12:56:40.185300 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 12:56:40.185350 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 12:56:40.185410 27203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:40.182767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56820 (local address 127.24.83.1:42947)
0806 12:56:40.182820 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:40.182822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:40.182836 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:56:40.182976 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:40.182977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:40.182997 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:40.183065 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:40.183234 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.183727 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.184557 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.184750 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.184857 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:40.185075 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:40.185179 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:40.185270 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:40.185303 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 12:56:40.185327 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:56:40.187351 27133 raft_consensus.cc:1261] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Refusing update from remote peer 964bbf2a9f984c71820f3b340debf1a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:40.187402 26990 raft_consensus.cc:1261] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Refusing update from remote peer 964bbf2a9f984c71820f3b340debf1a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:40.187749 27213 consensus_queue.cc:1035] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:40.187881 27215 consensus_queue.cc:1035] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:40.189218 27212 tablet_replica.cc:439] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373: TxnStatusTablet state changed. Reason: New leader 964bbf2a9f984c71820f3b340debf1a6. Latest consensus state: current_term: 1 leader_uuid: "964bbf2a9f984c71820f3b340debf1a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } }
I20260806 12:56:40.189322 27212 tablet_replica.cc:442] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:40.189500 27211 tablet_replica.cc:439] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: TxnStatusTablet state changed. Reason: New leader 964bbf2a9f984c71820f3b340debf1a6. Latest consensus state: current_term: 1 leader_uuid: "964bbf2a9f984c71820f3b340debf1a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } }
I20260806 12:56:40.189620 27211 tablet_replica.cc:442] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:40.190436 27215 tablet_replica.cc:439] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "964bbf2a9f984c71820f3b340debf1a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } }
I20260806 12:56:40.190523 27215 tablet_replica.cc:442] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:56:40.190989 27215 tablet_replica.cc:439] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "964bbf2a9f984c71820f3b340debf1a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } }
I20260806 12:56:40.191069 27215 tablet_replica.cc:442] T 69cafa0914ea4623bdd28f61fd9b5253 P 964bbf2a9f984c71820f3b340debf1a6: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:56:40.191282 27212 tablet_replica.cc:439] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "964bbf2a9f984c71820f3b340debf1a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } }
I20260806 12:56:40.191362 27212 tablet_replica.cc:442] T 69cafa0914ea4623bdd28f61fd9b5253 P 505a820efa3e45b484a44a0daf95f373: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:40.191460 27233 mvcc.cc:205] Tried to move back new op lower bound from 7315542016764653568 to 7315542016665071616. Current Snapshot: MvccSnapshot[applied={T|T < 7315542016764653568}]
I20260806 12:56:40.191622 27211 tablet_replica.cc:439] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "964bbf2a9f984c71820f3b340debf1a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } } }
I20260806 12:56:40.191715 27211 tablet_replica.cc:442] T 69cafa0914ea4623bdd28f61fd9b5253 P 1e7a022b41c8447abb35f2782a0ea165: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:56:40.192875 27232 mvcc.cc:205] Tried to move back new op lower bound from 7315542016764653568 to 7315542016665071616. Current Snapshot: MvccSnapshot[applied={T|T < 7315542016764653568 or (T in {7315542016764653568})}]
I20260806 12:56:40.195792 27203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:40.193271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56822 (local address 127.24.83.1:42947)
0806 12:56:40.193346 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:40.193348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:40.193359 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:56:40.193440 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:40.193440 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:40.193448 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:40.193487 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:40.193611 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.194058 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.195173 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.195355 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.195405 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:40.195582 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:40.195606 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:40.195644 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:40.195707 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 12:56:40.195723 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 12:56:40.200912 27203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:40.198278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:50647 (local address 127.24.83.1:42947)
0806 12:56:40.198328 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:40.198330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:40.198439 (+ 109us) server_negotiation.cc:407] Connection header received
0806 12:56:40.198491 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:40.198492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:40.198499 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:40.198543 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:40.198667 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.199087 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.200236 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.200427 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.200481 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:40.200720 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:40.200744 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:40.200782 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:40.200851 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 12:56:40.200865 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 12:56:40.201016 27242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:40.198194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:42947 (local address 127.24.83.1:50647)
0806 12:56:40.198356 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:40.198367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:40.198435 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:40.198546 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:40.198547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:40.198548 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:56:40.198612 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:40.198614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.199155 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.199155 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:40.200139 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:40.200142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:40.200516 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:40.200517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:40.200600 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:40.200617 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:56:40.200802 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:56:40.200804 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 12:56:40.200851 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 12:56:40.200897 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260806 12:56:40.203217 27133 raft_consensus.cc:1261] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Refusing update from remote peer 964bbf2a9f984c71820f3b340debf1a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:40.203308 26990 raft_consensus.cc:1261] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Refusing update from remote peer 964bbf2a9f984c71820f3b340debf1a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:40.203465 27215 consensus_queue.cc:1035] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:40.203585 27215 consensus_queue.cc:1035] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:40.204607 27231 mvcc.cc:205] Tried to move back new op lower bound from 7315542016830779392 to 7315542016077471744. Current Snapshot: MvccSnapshot[applied={T|T < 7315542016830779392}]
I20260806 12:56:40.215669 26683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:40.215783 26683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:40.216602 26683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41: Sending DeleteTablet for 3 replicas of tablet ed95f9454b9a4038818a21dea4834e1c
I20260806 12:56:40.216943 27113 tablet_service.cc:1514] Processing DeleteTablet for tablet ed95f9454b9a4038818a21dea4834e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:40 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:39634
I20260806 12:56:40.216943 26826 tablet_service.cc:1514] Processing DeleteTablet for tablet ed95f9454b9a4038818a21dea4834e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:40 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56814
I20260806 12:56:40.217203 27248 tablet_replica.cc:330] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: stopping tablet replica
I20260806 12:56:40.217258 26969 tablet_service.cc:1514] Processing DeleteTablet for tablet ed95f9454b9a4038818a21dea4834e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:40 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:40216
I20260806 12:56:40.217415 27248 raft_consensus.cc:2229] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:56:40.217443 27249 tablet_replica.cc:330] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: stopping tablet replica
I20260806 12:56:40.217428 26683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 12:56:40.217535 27249 raft_consensus.cc:2229] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 12:56:40.217574 26683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:56:40.217588 27249 raft_consensus.cc:2258] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:40.218075 27248 raft_consensus.cc:2258] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:40.218106 27249 ts_tablet_manager.cc:1905] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:40.218297 27247 tablet_replica.cc:330] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165: stopping tablet replica
I20260806 12:56:40.218446 27247 raft_consensus.cc:2229] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:40.218549 27247 raft_consensus.cc:2258] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:40.218902 26826 tablet_service.cc:1467] Processing CreateTablet for tablet 67fa6bd4a41142b6b8934c853a96914f (DEFAULT_TABLE table=test-table [id=17e7f6037376424f9efac658534e9858]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:40.219030 26826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67fa6bd4a41142b6b8934c853a96914f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:40.219123 27247 ts_tablet_manager.cc:1905] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:40.220301 27249 ts_tablet_manager.cc:1918] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:56:40.220386 27249 log.cc:1196] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/ed95f9454b9a4038818a21dea4834e1c
I20260806 12:56:40.220527 27206 tablet_bootstrap.cc:492] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6: Bootstrap starting.
I20260806 12:56:40.220672 27249 ts_tablet_manager.cc:1939] T ed95f9454b9a4038818a21dea4834e1c P 505a820efa3e45b484a44a0daf95f373: Deleting consensus metadata
I20260806 12:56:40.221092 27206 tablet_bootstrap.cc:654] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:40.221210 27247 ts_tablet_manager.cc:1918] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:56:40.221318 27247 log.cc:1196] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/ed95f9454b9a4038818a21dea4834e1c
I20260806 12:56:40.221683 27247 ts_tablet_manager.cc:1939] T ed95f9454b9a4038818a21dea4834e1c P 1e7a022b41c8447abb35f2782a0ea165: Deleting consensus metadata
I20260806 12:56:40.221665 26670 catalog_manager.cc:4841] TS 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801): tablet ed95f9454b9a4038818a21dea4834e1c (table test-table [id=7b83261499064fde99eeb265f8c34289]) successfully deleted
I20260806 12:56:40.221943 27113 tablet_service.cc:1467] Processing CreateTablet for tablet 67fa6bd4a41142b6b8934c853a96914f (DEFAULT_TABLE table=test-table [id=17e7f6037376424f9efac658534e9858]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:40.221943 26969 tablet_service.cc:1467] Processing CreateTablet for tablet 67fa6bd4a41142b6b8934c853a96914f (DEFAULT_TABLE table=test-table [id=17e7f6037376424f9efac658534e9858]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:40.222051 26969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67fa6bd4a41142b6b8934c853a96914f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:40.222051 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67fa6bd4a41142b6b8934c853a96914f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:40.223327 27204 tablet_bootstrap.cc:492] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165: Bootstrap starting.
I20260806 12:56:40.223690 27204 tablet_bootstrap.cc:654] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:40.223994 27205 tablet_bootstrap.cc:492] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373: Bootstrap starting.
I20260806 12:56:40.224329 27204 tablet_bootstrap.cc:492] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165: No bootstrap required, opened a new log
I20260806 12:56:40.224341 27205 tablet_bootstrap.cc:654] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:40.224392 27204 ts_tablet_manager.cc:1397] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 12:56:40.224543 27204 raft_consensus.cc:348] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.224619 27204 raft_consensus.cc:374] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:40.224647 27204 raft_consensus.cc:729] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7a022b41c8447abb35f2782a0ea165, State: Initialized, Role: FOLLOWER
I20260806 12:56:40.224689 27204 consensus_queue.cc:260] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.224782 27204 ts_tablet_manager.cc:1428] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 12:56:40.224910 27206 tablet_bootstrap.cc:492] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6: No bootstrap required, opened a new log
I20260806 12:56:40.224965 27206 ts_tablet_manager.cc:1397] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 12:56:40.225107 27206 raft_consensus.cc:348] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.225181 27206 raft_consensus.cc:374] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:40.225203 27206 raft_consensus.cc:729] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964bbf2a9f984c71820f3b340debf1a6, State: Initialized, Role: FOLLOWER
I20260806 12:56:40.225238 27206 consensus_queue.cc:260] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.225311 27206 ts_tablet_manager.cc:1428] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 12:56:40.225407 27205 tablet_bootstrap.cc:492] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373: No bootstrap required, opened a new log
I20260806 12:56:40.225451 27205 ts_tablet_manager.cc:1397] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 12:56:40.225575 26670 catalog_manager.cc:4841] TS 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467): tablet ed95f9454b9a4038818a21dea4834e1c (table test-table [id=7b83261499064fde99eeb265f8c34289]) successfully deleted
I20260806 12:56:40.225610 27205 raft_consensus.cc:348] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.225675 27205 raft_consensus.cc:374] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:40.225693 27205 raft_consensus.cc:729] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 505a820efa3e45b484a44a0daf95f373, State: Initialized, Role: FOLLOWER
I20260806 12:56:40.225730 27205 consensus_queue.cc:260] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.225827 27205 ts_tablet_manager.cc:1428] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 12:56:40.226168 27248 ts_tablet_manager.cc:1905] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:40.227617 27248 ts_tablet_manager.cc:1918] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:56:40.227689 27248 log.cc:1196] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/wal/wals/ed95f9454b9a4038818a21dea4834e1c
I20260806 12:56:40.227965 27248 ts_tablet_manager.cc:1939] T ed95f9454b9a4038818a21dea4834e1c P 964bbf2a9f984c71820f3b340debf1a6: Deleting consensus metadata
I20260806 12:56:40.228346 26668 catalog_manager.cc:4841] TS 964bbf2a9f984c71820f3b340debf1a6 (127.24.83.1:42947): tablet ed95f9454b9a4038818a21dea4834e1c (table test-table [id=7b83261499064fde99eeb265f8c34289]) successfully deleted
I20260806 12:56:40.261680 27213 raft_consensus.cc:482] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:56:40.261765 27213 raft_consensus.cc:504] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.261912 27213 leader_election.cc:288] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801), 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467)
I20260806 12:56:40.262217 27133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67fa6bd4a41142b6b8934c853a96914f" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7a022b41c8447abb35f2782a0ea165" is_pre_election: true
I20260806 12:56:40.262214 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67fa6bd4a41142b6b8934c853a96914f" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505a820efa3e45b484a44a0daf95f373" is_pre_election: true
I20260806 12:56:40.262324 26990 raft_consensus.cc:2454] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 0.
I20260806 12:56:40.262331 27133 raft_consensus.cc:2454] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 0.
I20260806 12:56:40.262524 26777 leader_election.cc:302] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 505a820efa3e45b484a44a0daf95f373, 964bbf2a9f984c71820f3b340debf1a6; no voters:
I20260806 12:56:40.262643 27213 raft_consensus.cc:2781] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:56:40.262696 27213 raft_consensus.cc:482] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:56:40.262724 27213 raft_consensus.cc:3037] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:40.263486 27213 raft_consensus.cc:504] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.263622 27213 leader_election.cc:288] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 election: Requested vote from peers 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801), 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467)
I20260806 12:56:40.263793 27133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67fa6bd4a41142b6b8934c853a96914f" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7a022b41c8447abb35f2782a0ea165"
I20260806 12:56:40.263800 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67fa6bd4a41142b6b8934c853a96914f" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505a820efa3e45b484a44a0daf95f373"
I20260806 12:56:40.263866 27133 raft_consensus.cc:3037] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:40.263866 26990 raft_consensus.cc:3037] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:40.264715 27133 raft_consensus.cc:2454] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 1.
I20260806 12:56:40.264715 26990 raft_consensus.cc:2454] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 1.
I20260806 12:56:40.264885 26777 leader_election.cc:302] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 505a820efa3e45b484a44a0daf95f373, 964bbf2a9f984c71820f3b340debf1a6; no voters:
I20260806 12:56:40.264976 27213 raft_consensus.cc:2781] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:40.265031 27213 raft_consensus.cc:686] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 1 LEADER]: Becoming Leader. State: Replica: 964bbf2a9f984c71820f3b340debf1a6, State: Running, Role: LEADER
I20260806 12:56:40.265075 27213 consensus_queue.cc:237] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:40.265681 26682 catalog_manager.cc:5495] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 964bbf2a9f984c71820f3b340debf1a6 (127.24.83.1). New cstate: current_term: 1 leader_uuid: "964bbf2a9f984c71820f3b340debf1a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } health_report { overall_health: UNKNOWN } } }
I20260806 12:56:40.276786 26990 raft_consensus.cc:1261] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Refusing update from remote peer 964bbf2a9f984c71820f3b340debf1a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:40.276847 27132 raft_consensus.cc:1261] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Refusing update from remote peer 964bbf2a9f984c71820f3b340debf1a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:40.277143 27215 consensus_queue.cc:1035] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:40.277314 27245 consensus_queue.cc:1035] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 12:56:40.282378 27231 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 12:56:41.800928 26682 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:41.801026 26682 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:41.801913 26682 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41: Sending DeleteTablet for 3 replicas of tablet 67fa6bd4a41142b6b8934c853a96914f
I20260806 12:56:41.802222 26969 tablet_service.cc:1514] Processing DeleteTablet for tablet 67fa6bd4a41142b6b8934c853a96914f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:41 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:40216
I20260806 12:56:41.802284 26825 tablet_service.cc:1514] Processing DeleteTablet for tablet 67fa6bd4a41142b6b8934c853a96914f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:41 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56814
I20260806 12:56:41.802317 27113 tablet_service.cc:1514] Processing DeleteTablet for tablet 67fa6bd4a41142b6b8934c853a96914f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:41 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:39634
I20260806 12:56:41.802482 27276 tablet_replica.cc:330] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6: stopping tablet replica
I20260806 12:56:41.802549 27276 raft_consensus.cc:2229] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:56:41.802616 27276 raft_consensus.cc:2258] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:41.802719 27275 tablet_replica.cc:330] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373: stopping tablet replica
I20260806 12:56:41.802781 27275 raft_consensus.cc:2229] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:41.802825 27275 raft_consensus.cc:2258] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:41.803052 27276 ts_tablet_manager.cc:1905] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:41.805065 27276 ts_tablet_manager.cc:1918] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 12:56:41.805138 27276 log.cc:1196] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/wal/wals/67fa6bd4a41142b6b8934c853a96914f
I20260806 12:56:41.805363 27276 ts_tablet_manager.cc:1939] T 67fa6bd4a41142b6b8934c853a96914f P 964bbf2a9f984c71820f3b340debf1a6: Deleting consensus metadata
I20260806 12:56:41.805599 27275 ts_tablet_manager.cc:1905] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:41.805794 26668 catalog_manager.cc:4841] TS 964bbf2a9f984c71820f3b340debf1a6 (127.24.83.1:42947): tablet 67fa6bd4a41142b6b8934c853a96914f (table test-table [id=17e7f6037376424f9efac658534e9858]) successfully deleted
I20260806 12:56:41.806859 27275 ts_tablet_manager.cc:1918] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 12:56:41.806941 27275 log.cc:1196] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/67fa6bd4a41142b6b8934c853a96914f
I20260806 12:56:41.807228 27275 ts_tablet_manager.cc:1939] T 67fa6bd4a41142b6b8934c853a96914f P 505a820efa3e45b484a44a0daf95f373: Deleting consensus metadata
I20260806 12:56:41.807521 27277 tablet_replica.cc:330] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165: stopping tablet replica
I20260806 12:56:41.807554 26670 catalog_manager.cc:4841] TS 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801): tablet 67fa6bd4a41142b6b8934c853a96914f (table test-table [id=17e7f6037376424f9efac658534e9858]) successfully deleted
I20260806 12:56:41.807595 27277 raft_consensus.cc:2229] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:41.807637 27277 raft_consensus.cc:2258] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:41.807876 27277 ts_tablet_manager.cc:1905] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:41.809425 27277 ts_tablet_manager.cc:1918] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 12:56:41.809479 27277 log.cc:1196] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/67fa6bd4a41142b6b8934c853a96914f
I20260806 12:56:41.809689 27277 ts_tablet_manager.cc:1939] T 67fa6bd4a41142b6b8934c853a96914f P 1e7a022b41c8447abb35f2782a0ea165: Deleting consensus metadata
I20260806 12:56:41.810106 26670 catalog_manager.cc:4841] TS 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467): tablet 67fa6bd4a41142b6b8934c853a96914f (table test-table [id=17e7f6037376424f9efac658534e9858]) successfully deleted
Aug 06 12:56:41 dist-test-slave-zkpd krb5kdc[26639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021001, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 12:56:41 dist-test-slave-zkpd krb5kdc[26639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021001, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:41.825929 27285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:41.818728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35464 (local address 127.24.83.62:44739)
0806 12:56:41.818868 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:41.818870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:41.818878 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:56:41.818925 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:41.818926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:41.818936 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:41.818995 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:41.819548 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:41.820116 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:41.820935 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:41.821130 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:41.821183 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:41.823433 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:41.823435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:41.823435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:41.823441 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:41.824984 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:41.825277 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:41.825278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:41.825278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:41.825329 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:41.825486 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:41.825487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:41.825487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:41.825597 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:41.825675 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:41.825765 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 12:56:41.825828 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260806 12:56:41.827425 26682 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:56:41.827593 26682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:56:41.829100 27113 tablet_service.cc:1467] Processing CreateTablet for tablet f64143b8f1bd47e98affdf062367014d (DEFAULT_TABLE table=test-table [id=8028b9625930487b8bd508472eba25c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:41.829094 26825 tablet_service.cc:1467] Processing CreateTablet for tablet f64143b8f1bd47e98affdf062367014d (DEFAULT_TABLE table=test-table [id=8028b9625930487b8bd508472eba25c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:41.829120 26969 tablet_service.cc:1467] Processing CreateTablet for tablet f64143b8f1bd47e98affdf062367014d (DEFAULT_TABLE table=test-table [id=8028b9625930487b8bd508472eba25c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:41.829231 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64143b8f1bd47e98affdf062367014d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:41.829231 26825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64143b8f1bd47e98affdf062367014d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:41.829265 26969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64143b8f1bd47e98affdf062367014d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:41.830336 27286 tablet_bootstrap.cc:492] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373: Bootstrap starting.
I20260806 12:56:41.830667 27287 tablet_bootstrap.cc:492] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165: Bootstrap starting.
I20260806 12:56:41.830806 27288 tablet_bootstrap.cc:492] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6: Bootstrap starting.
I20260806 12:56:41.830929 27286 tablet_bootstrap.cc:654] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:41.831198 27288 tablet_bootstrap.cc:654] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:41.831377 27287 tablet_bootstrap.cc:654] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:41.831779 27288 tablet_bootstrap.cc:492] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6: No bootstrap required, opened a new log
I20260806 12:56:41.831815 27286 tablet_bootstrap.cc:492] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373: No bootstrap required, opened a new log
I20260806 12:56:41.831841 27288 ts_tablet_manager.cc:1397] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:41.831879 27286 ts_tablet_manager.cc:1397] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:56:41.832021 27287 tablet_bootstrap.cc:492] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165: No bootstrap required, opened a new log
I20260806 12:56:41.831988 27288 raft_consensus.cc:348] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.832065 27288 raft_consensus.cc:374] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:41.832073 27287 ts_tablet_manager.cc:1397] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:41.832039 27286 raft_consensus.cc:348] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.832098 27288 raft_consensus.cc:729] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964bbf2a9f984c71820f3b340debf1a6, State: Initialized, Role: FOLLOWER
I20260806 12:56:41.832109 27286 raft_consensus.cc:374] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:41.832140 27286 raft_consensus.cc:729] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 505a820efa3e45b484a44a0daf95f373, State: Initialized, Role: FOLLOWER
I20260806 12:56:41.832145 27288 consensus_queue.cc:260] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.832175 27287 raft_consensus.cc:348] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.832203 27286 consensus_queue.cc:260] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.832237 27287 raft_consensus.cc:374] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:41.832235 27288 ts_tablet_manager.cc:1428] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:41.832260 27287 raft_consensus.cc:729] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7a022b41c8447abb35f2782a0ea165, State: Initialized, Role: FOLLOWER
I20260806 12:56:41.832330 27287 consensus_queue.cc:260] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.832360 27286 ts_tablet_manager.cc:1428] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:41.832521 27287 ts_tablet_manager.cc:1428] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 12:56:41.913102 27258 raft_consensus.cc:482] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:56:41.913183 27258 raft_consensus.cc:504] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.913334 27258 leader_election.cc:288] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801), 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467)
I20260806 12:56:41.913583 27133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64143b8f1bd47e98affdf062367014d" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7a022b41c8447abb35f2782a0ea165" is_pre_election: true
I20260806 12:56:41.913583 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64143b8f1bd47e98affdf062367014d" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505a820efa3e45b484a44a0daf95f373" is_pre_election: true
I20260806 12:56:41.913708 26990 raft_consensus.cc:2454] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 0.
I20260806 12:56:41.913708 27133 raft_consensus.cc:2454] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 0.
I20260806 12:56:41.913872 26777 leader_election.cc:302] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7a022b41c8447abb35f2782a0ea165, 964bbf2a9f984c71820f3b340debf1a6; no voters:
I20260806 12:56:41.913997 27258 raft_consensus.cc:2781] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:56:41.914037 27258 raft_consensus.cc:482] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:56:41.914060 27258 raft_consensus.cc:3037] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:41.914783 27258 raft_consensus.cc:504] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.914929 27258 leader_election.cc:288] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 election: Requested vote from peers 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801), 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467)
I20260806 12:56:41.915066 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64143b8f1bd47e98affdf062367014d" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505a820efa3e45b484a44a0daf95f373"
I20260806 12:56:41.915151 26990 raft_consensus.cc:3037] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:41.915118 27133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64143b8f1bd47e98affdf062367014d" candidate_uuid: "964bbf2a9f984c71820f3b340debf1a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7a022b41c8447abb35f2782a0ea165"
I20260806 12:56:41.915241 27133 raft_consensus.cc:3037] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:41.915767 26990 raft_consensus.cc:2454] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 1.
I20260806 12:56:41.916006 26777 leader_election.cc:302] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 505a820efa3e45b484a44a0daf95f373, 964bbf2a9f984c71820f3b340debf1a6; no voters:
I20260806 12:56:41.916019 27133 raft_consensus.cc:2454] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964bbf2a9f984c71820f3b340debf1a6 in term 1.
I20260806 12:56:41.916263 27258 raft_consensus.cc:2781] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:41.916347 27258 raft_consensus.cc:686] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 1 LEADER]: Becoming Leader. State: Replica: 964bbf2a9f984c71820f3b340debf1a6, State: Running, Role: LEADER
I20260806 12:56:41.916410 27258 consensus_queue.cc:237] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.917069 26683 catalog_manager.cc:5495] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 964bbf2a9f984c71820f3b340debf1a6 (127.24.83.1). New cstate: current_term: 1 leader_uuid: "964bbf2a9f984c71820f3b340debf1a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } health_report { overall_health: UNKNOWN } } }
I20260806 12:56:41.926595 27296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:41.924168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54872 (local address 127.24.83.1:42947)
0806 12:56:41.924332 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:41.924334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:41.924341 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:56:41.924367 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:41.924368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:41.924377 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:41.924452 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:41.924588 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:41.925085 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:41.925864 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:41.926092 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:41.926175 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:41.926337 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:41.926370 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:41.926434 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:41.926487 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 12:56:41.926510 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260806 12:56:41.928712 26990 raft_consensus.cc:1261] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Refusing update from remote peer 964bbf2a9f984c71820f3b340debf1a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:41.928753 27133 raft_consensus.cc:1261] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Refusing update from remote peer 964bbf2a9f984c71820f3b340debf1a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:41.929003 27213 consensus_queue.cc:1035] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:41.929088 27258 consensus_queue.cc:1035] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:41.938879 26683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:41.938970 26683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:41.939589 26683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41: Sending DeleteTablet for 3 replicas of tablet f64143b8f1bd47e98affdf062367014d
I20260806 12:56:41.939805 26825 tablet_service.cc:1514] Processing DeleteTablet for tablet f64143b8f1bd47e98affdf062367014d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:41 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56814
I20260806 12:56:41.939924 27276 tablet_replica.cc:330] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6: stopping tablet replica
I20260806 12:56:41.939944 27113 tablet_service.cc:1514] Processing DeleteTablet for tablet f64143b8f1bd47e98affdf062367014d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:41 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:39634
I20260806 12:56:41.939986 27276 raft_consensus.cc:2229] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:56:41.940026 27276 raft_consensus.cc:2258] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:41.940055 27277 tablet_replica.cc:330] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165: stopping tablet replica
I20260806 12:56:41.940119 27277 raft_consensus.cc:2229] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:41.940161 27277 raft_consensus.cc:2258] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:41.940308 26969 tablet_service.cc:1514] Processing DeleteTablet for tablet f64143b8f1bd47e98affdf062367014d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:41 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:40216
I20260806 12:56:41.940383 27276 ts_tablet_manager.cc:1905] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:41.940410 27275 tablet_replica.cc:330] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373: stopping tablet replica
I20260806 12:56:41.940457 27275 raft_consensus.cc:2229] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:41.940497 27275 raft_consensus.cc:2258] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:41.940487 27277 ts_tablet_manager.cc:1905] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:41.940682 26683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:56:41.940817 26683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:56:41.941118 27275 ts_tablet_manager.cc:1905] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:41.942159 27277 ts_tablet_manager.cc:1918] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:56:41.942173 27276 ts_tablet_manager.cc:1918] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:56:41.942231 27276 log.cc:1196] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-0/wal/wals/f64143b8f1bd47e98affdf062367014d
I20260806 12:56:41.942233 27277 log.cc:1196] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/f64143b8f1bd47e98affdf062367014d
I20260806 12:56:41.942471 27277 ts_tablet_manager.cc:1939] T f64143b8f1bd47e98affdf062367014d P 1e7a022b41c8447abb35f2782a0ea165: Deleting consensus metadata
I20260806 12:56:41.942498 27276 ts_tablet_manager.cc:1939] T f64143b8f1bd47e98affdf062367014d P 964bbf2a9f984c71820f3b340debf1a6: Deleting consensus metadata
I20260806 12:56:41.942713 27275 ts_tablet_manager.cc:1918] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:56:41.942770 27275 log.cc:1196] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/f64143b8f1bd47e98affdf062367014d
I20260806 12:56:41.942842 26668 catalog_manager.cc:4841] TS 964bbf2a9f984c71820f3b340debf1a6 (127.24.83.1:42947): tablet f64143b8f1bd47e98affdf062367014d (table test-table [id=8028b9625930487b8bd508472eba25c0]) successfully deleted
I20260806 12:56:41.942998 27275 ts_tablet_manager.cc:1939] T f64143b8f1bd47e98affdf062367014d P 505a820efa3e45b484a44a0daf95f373: Deleting consensus metadata
I20260806 12:56:41.943171 26670 catalog_manager.cc:4841] TS 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467): tablet f64143b8f1bd47e98affdf062367014d (table test-table [id=8028b9625930487b8bd508472eba25c0]) successfully deleted
I20260806 12:56:41.943415 26670 catalog_manager.cc:4841] TS 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801): tablet f64143b8f1bd47e98affdf062367014d (table test-table [id=8028b9625930487b8bd508472eba25c0]) successfully deleted
I20260806 12:56:41.944269 26825 tablet_service.cc:1467] Processing CreateTablet for tablet 49d5e7a3de704eb4bdd4b06e45a00203 (DEFAULT_TABLE table=test-table [id=c52a816875b64c03a716cb288e092e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:41.944315 26969 tablet_service.cc:1467] Processing CreateTablet for tablet 49d5e7a3de704eb4bdd4b06e45a00203 (DEFAULT_TABLE table=test-table [id=c52a816875b64c03a716cb288e092e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:41.944348 27113 tablet_service.cc:1467] Processing CreateTablet for tablet 49d5e7a3de704eb4bdd4b06e45a00203 (DEFAULT_TABLE table=test-table [id=c52a816875b64c03a716cb288e092e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:41.944382 26825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49d5e7a3de704eb4bdd4b06e45a00203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:41.944408 26969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49d5e7a3de704eb4bdd4b06e45a00203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:41.944480 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49d5e7a3de704eb4bdd4b06e45a00203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:41.945282 27287 tablet_bootstrap.cc:492] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165: Bootstrap starting.
I20260806 12:56:41.945554 27288 tablet_bootstrap.cc:492] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6: Bootstrap starting.
I20260806 12:56:41.945632 27287 tablet_bootstrap.cc:654] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:41.945866 27288 tablet_bootstrap.cc:654] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:41.946130 27286 tablet_bootstrap.cc:492] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373: Bootstrap starting.
I20260806 12:56:41.946261 27287 tablet_bootstrap.cc:492] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165: No bootstrap required, opened a new log
I20260806 12:56:41.946314 27287 ts_tablet_manager.cc:1397] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 12:56:41.946398 27288 tablet_bootstrap.cc:492] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6: No bootstrap required, opened a new log
I20260806 12:56:41.946437 27288 ts_tablet_manager.cc:1397] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:41.946470 27287 raft_consensus.cc:348] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.946537 27287 raft_consensus.cc:374] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:41.946558 27287 raft_consensus.cc:729] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7a022b41c8447abb35f2782a0ea165, State: Initialized, Role: FOLLOWER
I20260806 12:56:41.946555 27288 raft_consensus.cc:348] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.946614 27288 raft_consensus.cc:374] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:41.946636 27288 raft_consensus.cc:729] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964bbf2a9f984c71820f3b340debf1a6, State: Initialized, Role: FOLLOWER
I20260806 12:56:41.946605 27287 consensus_queue.cc:260] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.946676 27286 tablet_bootstrap.cc:654] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:41.946677 27288 consensus_queue.cc:260] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.946707 27287 ts_tablet_manager.cc:1428] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:41.946767 27288 ts_tablet_manager.cc:1428] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:41.947407 27286 tablet_bootstrap.cc:492] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373: No bootstrap required, opened a new log
I20260806 12:56:41.947456 27286 ts_tablet_manager.cc:1397] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:41.947590 27286 raft_consensus.cc:348] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.947662 27286 raft_consensus.cc:374] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:41.947677 27286 raft_consensus.cc:729] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 505a820efa3e45b484a44a0daf95f373, State: Initialized, Role: FOLLOWER
I20260806 12:56:41.947727 27286 consensus_queue.cc:260] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:41.947829 27286 ts_tablet_manager.cc:1428] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:42.239338 27293 raft_consensus.cc:482] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:56:42.239430 27293 raft_consensus.cc:504] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:42.239755 27293 leader_election.cc:288] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 964bbf2a9f984c71820f3b340debf1a6 (127.24.83.1:42947), 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801)
I20260806 12:56:42.242947 27296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:42.240023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:47439 (local address 127.24.83.1:42947)
0806 12:56:42.240072 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:42.240075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:42.240202 (+ 127us) server_negotiation.cc:407] Connection header received
0806 12:56:42.240327 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:42.240328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:42.240347 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:42.240414 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:42.240560 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.241064 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.242199 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.242568 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.242661 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:42.242672 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:42.242834 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 12:56:42.242875 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:56:42.242981 27305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:42.239944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:42947 (local address 127.24.83.3:47439)
0806 12:56:42.240134 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:42.240144 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:42.240227 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:42.240417 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:42.240418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:42.240421 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:42.240485 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:42.240487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.241178 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.241179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:42.242098 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:42.242101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.242663 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.242663 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:42.242751 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:42.242754 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:56:42.242828 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 12:56:42.242869 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":44,"thread_start_us":111,"threads_started":1}
I20260806 12:56:42.242987 27304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:42.239986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:37801 (local address 127.24.83.3:54179)
0806 12:56:42.240131 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:42.240144 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:42.240208 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:42.240432 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:42.240433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:42.240436 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:42.240511 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:42.240513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.241220 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.241221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:42.242096 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:42.242099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.242682 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.242683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:42.242752 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:42.242755 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:56:42.242834 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 12:56:42.242875 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":25,"thread_start_us":100,"threads_started":1}
I20260806 12:56:42.242965 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:42.240094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:54179 (local address 127.24.83.2:37801)
0806 12:56:42.240229 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:42.240232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:42.240241 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:56:42.240272 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:42.240273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:42.240292 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:42.240409 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:42.240588 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.241088 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.242194 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.242598 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.242682 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:42.242692 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:42.242842 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 12:56:42.242876 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":51,"threads_started":1}
I20260806 12:56:42.243274 26846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49d5e7a3de704eb4bdd4b06e45a00203" candidate_uuid: "1e7a022b41c8447abb35f2782a0ea165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964bbf2a9f984c71820f3b340debf1a6" is_pre_election: true
I20260806 12:56:42.243446 26846 raft_consensus.cc:2454] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e7a022b41c8447abb35f2782a0ea165 in term 0.
I20260806 12:56:42.243544 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49d5e7a3de704eb4bdd4b06e45a00203" candidate_uuid: "1e7a022b41c8447abb35f2782a0ea165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505a820efa3e45b484a44a0daf95f373" is_pre_election: true
I20260806 12:56:42.243671 26989 raft_consensus.cc:2454] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e7a022b41c8447abb35f2782a0ea165 in term 0.
I20260806 12:56:42.243678 27061 leader_election.cc:302] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7a022b41c8447abb35f2782a0ea165, 964bbf2a9f984c71820f3b340debf1a6; no voters:
I20260806 12:56:42.243836 27293 raft_consensus.cc:2781] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:56:42.243891 27293 raft_consensus.cc:482] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:56:42.243947 27293 raft_consensus.cc:3037] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:42.244779 27293 raft_consensus.cc:504] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:42.244943 27293 leader_election.cc:288] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [CANDIDATE]: Term 1 election: Requested vote from peers 964bbf2a9f984c71820f3b340debf1a6 (127.24.83.1:42947), 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801)
I20260806 12:56:42.245075 26846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49d5e7a3de704eb4bdd4b06e45a00203" candidate_uuid: "1e7a022b41c8447abb35f2782a0ea165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964bbf2a9f984c71820f3b340debf1a6"
I20260806 12:56:42.245152 26846 raft_consensus.cc:3037] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:42.245108 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49d5e7a3de704eb4bdd4b06e45a00203" candidate_uuid: "1e7a022b41c8447abb35f2782a0ea165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505a820efa3e45b484a44a0daf95f373"
I20260806 12:56:42.245198 26989 raft_consensus.cc:3037] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:42.245836 26846 raft_consensus.cc:2454] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e7a022b41c8447abb35f2782a0ea165 in term 1.
I20260806 12:56:42.245844 26989 raft_consensus.cc:2454] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e7a022b41c8447abb35f2782a0ea165 in term 1.
I20260806 12:56:42.246016 27064 leader_election.cc:302] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7a022b41c8447abb35f2782a0ea165, 505a820efa3e45b484a44a0daf95f373; no voters:
I20260806 12:56:42.246112 27293 raft_consensus.cc:2781] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:42.246286 27293 raft_consensus.cc:686] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 1 LEADER]: Becoming Leader. State: Replica: 1e7a022b41c8447abb35f2782a0ea165, State: Running, Role: LEADER
I20260806 12:56:42.246383 27293 consensus_queue.cc:237] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } }
I20260806 12:56:42.247100 26683 catalog_manager.cc:5495] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3). New cstate: current_term: 1 leader_uuid: "1e7a022b41c8447abb35f2782a0ea165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e7a022b41c8447abb35f2782a0ea165" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46467 } health_report { overall_health: HEALTHY } } }
I20260806 12:56:42.305105 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:42.302716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.24.83.3:46467)
0806 12:56:42.302914 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:42.302915 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:42.302922 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:56:42.302959 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:42.302960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:42.302980 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:42.303048 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:42.303152 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.303626 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.304371 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.304570 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.304622 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:42.304762 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:42.304865 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:42.304928 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:42.304965 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 12:56:42.304981 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":129,"threads_started":1}
I20260806 12:56:42.310067 27296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:42.307523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:39555 (local address 127.24.83.1:42947)
0806 12:56:42.307592 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:42.307594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:42.307714 (+ 120us) server_negotiation.cc:407] Connection header received
0806 12:56:42.307777 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:42.307778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:42.307790 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:42.307845 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:42.308019 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.308474 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.309336 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.309532 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.309604 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:42.309809 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:42.309846 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:42.309907 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:42.309969 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 12:56:42.309991 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:56:42.310097 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:42.307434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:42947 (local address 127.24.83.3:39555)
0806 12:56:42.307627 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:42.307647 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:42.307724 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:42.307848 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:42.307849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:42.307850 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:56:42.307947 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:42.307949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.308565 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.308566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:42.309243 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:42.309245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.309621 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.309622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:42.309703 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:42.309716 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:56:42.309910 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:56:42.309912 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 12:56:42.309964 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 12:56:42.310002 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260806 12:56:42.312111 26989 raft_consensus.cc:1261] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Refusing update from remote peer 1e7a022b41c8447abb35f2782a0ea165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:42.312247 26846 raft_consensus.cc:1261] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6 [term 1 FOLLOWER]: Refusing update from remote peer 1e7a022b41c8447abb35f2782a0ea165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:42.312407 27307 consensus_queue.cc:1035] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "505a820efa3e45b484a44a0daf95f373" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:42.312559 27293 consensus_queue.cc:1035] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "964bbf2a9f984c71820f3b340debf1a6" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:42.319792 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:42.317523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:40007 (local address 127.24.83.3:46467)
0806 12:56:42.317559 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:42.317561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:42.317585 (+ 24us) server_negotiation.cc:407] Connection header received
0806 12:56:42.317606 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:42.317606 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:42.317615 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:42.317655 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:42.317774 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.318282 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.319129 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.319365 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.319424 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:42.319571 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:42.319602 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:42.319648 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:42.319705 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 12:56:42.319722 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 12:56:42.319819 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:42.317346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:46467 (local address 127.24.83.1:40007)
0806 12:56:42.317488 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:42.317498 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:42.317552 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:42.317668 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:42.317669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:42.317670 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:56:42.317714 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:42.317716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.318366 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.318367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:42.319005 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:42.319007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:42.319424 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:42.319425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:42.319480 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:42.319483 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:56:42.319648 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:56:42.319650 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 12:56:42.319700 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 12:56:42.319743 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
W20260806 12:56:42.320123 27260 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 12:56:43.843918 26682 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:43.844014 26682 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:43.844972 26682 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e6c29fedee94c65a4f346199331ae41: Sending DeleteTablet for 3 replicas of tablet 49d5e7a3de704eb4bdd4b06e45a00203
I20260806 12:56:43.845222 26825 tablet_service.cc:1514] Processing DeleteTablet for tablet 49d5e7a3de704eb4bdd4b06e45a00203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:43 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56814
I20260806 12:56:43.845268 26969 tablet_service.cc:1514] Processing DeleteTablet for tablet 49d5e7a3de704eb4bdd4b06e45a00203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:43 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:40216
I20260806 12:56:43.845256 27113 tablet_service.cc:1514] Processing DeleteTablet for tablet 49d5e7a3de704eb4bdd4b06e45a00203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:43 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:39634
I20260806 12:56:43.845372 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 26759
I20260806 12:56:43.845417 27340 tablet_replica.cc:330] T 49d5e7a3de704eb4bdd4b06e45a00203 P 964bbf2a9f984c71820f3b340debf1a6: stopping tablet replica
I20260806 12:56:43.847113 27343 tablet_replica.cc:330] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165: stopping tablet replica
I20260806 12:56:43.847177 27343 raft_consensus.cc:2229] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:56:43.847265 27343 raft_consensus.cc:2258] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:43.847535 27343 ts_tablet_manager.cc:1905] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:43.849395 27343 ts_tablet_manager.cc:1918] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 12:56:43.849447 27343 log.cc:1196] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/49d5e7a3de704eb4bdd4b06e45a00203
I20260806 12:56:43.849668 27343 ts_tablet_manager.cc:1939] T 49d5e7a3de704eb4bdd4b06e45a00203 P 1e7a022b41c8447abb35f2782a0ea165: Deleting consensus metadata
I20260806 12:56:43.849834 27341 tablet_replica.cc:330] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373: stopping tablet replica
I20260806 12:56:43.849896 27341 raft_consensus.cc:2229] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:43.849936 27341 raft_consensus.cc:2258] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:43.850176 26670 catalog_manager.cc:4841] TS 1e7a022b41c8447abb35f2782a0ea165 (127.24.83.3:46467): tablet 49d5e7a3de704eb4bdd4b06e45a00203 (table test-table [id=c52a816875b64c03a716cb288e092e4c]) successfully deleted
I20260806 12:56:43.850445 27341 ts_tablet_manager.cc:1905] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:43.851856 27341 ts_tablet_manager.cc:1918] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 12:56:43.851958 27341 log.cc:1196] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/49d5e7a3de704eb4bdd4b06e45a00203
I20260806 12:56:43.852166 27341 ts_tablet_manager.cc:1939] T 49d5e7a3de704eb4bdd4b06e45a00203 P 505a820efa3e45b484a44a0daf95f373: Deleting consensus metadata
I20260806 12:56:43.852468 26670 catalog_manager.cc:4841] TS 505a820efa3e45b484a44a0daf95f373 (127.24.83.2:37801): tablet 49d5e7a3de704eb4bdd4b06e45a00203 (table test-table [id=c52a816875b64c03a716cb288e092e4c]) successfully deleted
W20260806 12:56:43.856477 26668 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:42947 (error 108)
I20260806 12:56:43.856738 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 26902
I20260806 12:56:43.857626 27345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:43.856904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:42947 (local address 127.0.0.1:54876)
0806 12:56:43.857525 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:43.857577 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":59,"threads_started":1}
W20260806 12:56:43.857728 26668 catalog_manager.cc:4584] TS 964bbf2a9f984c71820f3b340debf1a6 (127.24.83.1:42947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 49d5e7a3de704eb4bdd4b06e45a00203: Network error: Client connection negotiation failed: client connection to 127.24.83.1:42947: connect: Connection refused (error 111)
I20260806 12:56:43.861912 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 27045
I20260806 12:56:43.868108 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 26652
2026-08-06T12:56:43Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5819 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:56:43 dist-test-slave-zkpd krb5kdc[27348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:56:43 dist-test-slave-zkpd krb5kdc[27348](info): set up 2 sockets
Aug 06 12:56:43 dist-test-slave-zkpd krb5kdc[27348](info): commencing operation
krb5kdc: starting...
W20260806 12:56:44.917689 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:56:44 dist-test-slave-zkpd krb5kdc[27348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:56:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:56:45.031941 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:37407
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:37407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:45.113569 27362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:45.113787 27362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:45.113811 27362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:45.114900 27362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:45.114934 27362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:45.114949 27362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:45.114961 27362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:45.114974 27362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:45.116477 27362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:37407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:37407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:45.116667 27362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:45.116845 27362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:45.121628 27367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:45.121603 27368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:45.121719 27362 server_base.cc:1034] running on GCE node
W20260806 12:56:45.121601 27370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:45.122098 27362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:45.122385 27362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:45.123533 27362 hybrid_clock.cc:648] HybridClock initialized: now 1786021005123511 us; error 36 us; skew 500 ppm
Aug 06 12:56:45 dist-test-slave-zkpd krb5kdc[27348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021005, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:45.126044 27362 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:56:45.126546 27362 webserver.cc:460] Webserver started at http://127.24.83.62:42569/ using document root <none> and password file <none>
I20260806 12:56:45.126782 27362 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:45.126829 27362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:45.126991 27362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:45.128106 27362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "bd45905ca55f4facb2b3e8770f500f1c"
format_stamp: "Formatted at 2026-08-06 12:56:45 on dist-test-slave-zkpd"
server_key: "b967a1b668bbbaf7d6e0b129c437d4c6"
server_key_iv: "07ae6466a484fe4f0e3256cc7db3810f"
server_key_version: "encryptionkey@0"
I20260806 12:56:45.128468 27362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "bd45905ca55f4facb2b3e8770f500f1c"
format_stamp: "Formatted at 2026-08-06 12:56:45 on dist-test-slave-zkpd"
server_key: "b967a1b668bbbaf7d6e0b129c437d4c6"
server_key_iv: "07ae6466a484fe4f0e3256cc7db3810f"
server_key_version: "encryptionkey@0"
I20260806 12:56:45.129757 27362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:56:45.130503 27377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:45.130684 27362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:45.130785 27362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "bd45905ca55f4facb2b3e8770f500f1c"
format_stamp: "Formatted at 2026-08-06 12:56:45 on dist-test-slave-zkpd"
server_key: "b967a1b668bbbaf7d6e0b129c437d4c6"
server_key_iv: "07ae6466a484fe4f0e3256cc7db3810f"
server_key_version: "encryptionkey@0"
I20260806 12:56:45.130865 27362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:45.160619 27362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:45.162539 27362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:45.162719 27362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:45.167130 27362 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:37407
I20260806 12:56:45.167169 27432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:37407 every 8 connection(s)
I20260806 12:56:45.167563 27362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:45.168500 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:45.170804 27433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c: Bootstrap starting.
I20260806 12:56:45.171380 27433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:45.171675 27433 log.cc:824] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:45.172421 27433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c: No bootstrap required, opened a new log
I20260806 12:56:45.173996 27433 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd45905ca55f4facb2b3e8770f500f1c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37407 } }
I20260806 12:56:45.174131 27433 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:45.174180 27433 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd45905ca55f4facb2b3e8770f500f1c, State: Initialized, Role: FOLLOWER
I20260806 12:56:45.174324 27433 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd45905ca55f4facb2b3e8770f500f1c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37407 } }
I20260806 12:56:45.174383 27433 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:45.174438 27433 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:45.174487 27433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:45.175024 27433 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd45905ca55f4facb2b3e8770f500f1c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37407 } }
I20260806 12:56:45.175135 27433 leader_election.cc:302] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd45905ca55f4facb2b3e8770f500f1c; no voters:
I20260806 12:56:45.175330 27433 leader_election.cc:288] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:45.175369 27436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:45.175545 27436 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [term 1 LEADER]: Becoming Leader. State: Replica: bd45905ca55f4facb2b3e8770f500f1c, State: Running, Role: LEADER
I20260806 12:56:45.175587 27433 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:45.175663 27436 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd45905ca55f4facb2b3e8770f500f1c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37407 } }
I20260806 12:56:45.176075 27437 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd45905ca55f4facb2b3e8770f500f1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd45905ca55f4facb2b3e8770f500f1c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37407 } } }
I20260806 12:56:45.176177 27437 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:45.176084 27438 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd45905ca55f4facb2b3e8770f500f1c. Latest consensus state: current_term: 1 leader_uuid: "bd45905ca55f4facb2b3e8770f500f1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd45905ca55f4facb2b3e8770f500f1c" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37407 } } }
I20260806 12:56:45.176415 27438 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:45.176472 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 27362
I20260806 12:56:45.176558 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:45.176730 24908 external_mini_cluster.cc:1428] Setting key 934d8b9c429190ddfcca9b03ee1dfeec
I20260806 12:56:45.177114 27440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:45.177687 27440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:45.179191 27440 catalog_manager.cc:1347] Generated new cluster ID: 8dc6ffad9fce40cab4889cbf94f2dce3
I20260806 12:56:45.179243 27440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:56:45 dist-test-slave-zkpd krb5kdc[27348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:45.189088 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:45.177424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60886 (local address 127.24.83.62:37407)
0806 12:56:45.178180 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:45.178191 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:45.178228 (+ 37us) server_negotiation.cc:407] Connection header received
0806 12:56:45.178345 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:45.178352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:45.178407 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:45.178548 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:45.179325 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:45.180803 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:45.181864 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:45.182483 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:45.182590 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:45.185441 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:45.185445 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:45.185446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:45.185458 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:45.187432 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:45.187846 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:45.187849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:45.187849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:45.187971 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:45.188193 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:45.188193 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:45.188194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:45.188500 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:45.188643 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:45.188747 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 12:56:45.188872 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":85,"threads_started":1}
I20260806 12:56:45.196066 27440 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:45.196593 27440 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:45.202569 27440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd45905ca55f4facb2b3e8770f500f1c: Generated new TSK 0
I20260806 12:56:45.202756 27440 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:56:45.228611 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:37407
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:45.317536 27460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:45.317783 27460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:45.317809 27460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:45.319046 27460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:45.319116 27460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:45.319164 27460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:56:45.320824 27460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:37407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:45.321081 27460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:45.321316 27460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:45.326588 27468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:45.326825 27466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:45.326599 27465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:45.326947 27460 server_base.cc:1034] running on GCE node
I20260806 12:56:45.327134 27460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:45.327348 27460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:45.328500 27460 hybrid_clock.cc:648] HybridClock initialized: now 1786021005328488 us; error 26 us; skew 500 ppm
Aug 06 12:56:45 dist-test-slave-zkpd krb5kdc[27348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021005, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:45.331066 27460 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:56:45.331453 27460 webserver.cc:460] Webserver started at http://127.24.83.1:43093/ using document root <none> and password file <none>
I20260806 12:56:45.331650 27460 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:45.331691 27460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:45.331784 27460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:45.332976 27460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "9ea4529583f640d2b761b64dc6dcdd9f"
format_stamp: "Formatted at 2026-08-06 12:56:45 on dist-test-slave-zkpd"
server_key: "b6e329acc25ba09a59e55ee16ac4c423"
server_key_iv: "216b7b71852f26aa31db13691177071d"
server_key_version: "encryptionkey@0"
I20260806 12:56:45.333312 27460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "9ea4529583f640d2b761b64dc6dcdd9f"
format_stamp: "Formatted at 2026-08-06 12:56:45 on dist-test-slave-zkpd"
server_key: "b6e329acc25ba09a59e55ee16ac4c423"
server_key_iv: "216b7b71852f26aa31db13691177071d"
server_key_version: "encryptionkey@0"
I20260806 12:56:45.334725 27460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:56:45.335455 27476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:45.335629 27460 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:45.335723 27460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "9ea4529583f640d2b761b64dc6dcdd9f"
format_stamp: "Formatted at 2026-08-06 12:56:45 on dist-test-slave-zkpd"
server_key: "b6e329acc25ba09a59e55ee16ac4c423"
server_key_iv: "216b7b71852f26aa31db13691177071d"
server_key_version: "encryptionkey@0"
I20260806 12:56:45.335777 27460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:45.346984 27460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:45.349514 27460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:45.349665 27460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:45.349860 27460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:45.350174 27460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:45.350207 27460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:45.350263 27460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:45.350298 27460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:45.356918 27460 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:37557
I20260806 12:56:45.356971 27593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:37557 every 8 connection(s)
I20260806 12:56:45.357420 27460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:56:45.363509 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 27460
I20260806 12:56:45.363590 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
Aug 06 12:56:45 dist-test-slave-zkpd krb5kdc[27348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021005, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:45.363782 24908 external_mini_cluster.cc:1428] Setting key 9cc90386e8718ab073cf74cb40eeee09
I20260806 12:56:45.367478 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:45.358895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:44625 (local address 127.24.83.62:37407)
0806 12:56:45.358949 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:45.358951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:45.358963 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:56:45.359401 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:45.359402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:45.359415 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:45.359463 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:45.360373 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:45.360879 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:45.361849 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:45.362039 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:45.362091 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:45.364356 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:45.364357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:45.364357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:45.364364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:45.366029 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:45.366536 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:45.366537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:45.366537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:45.366590 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:45.366866 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:45.366866 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:45.366867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:45.366972 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:45.367024 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:45.367345 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 12:56:45.367414 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:56:45.367764 27596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:45.358502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:37407 (local address 127.24.83.1:44625)
0806 12:56:45.358755 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:45.358805 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:45.359306 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:45.359527 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:45.359530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:45.360105 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:45.360294 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:45.360299 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:45.360961 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:45.360961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:45.361775 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:45.361777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:45.362110 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:45.362111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:45.362207 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:45.362746 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:45.362765 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:45.364275 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:45.366119 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:45.366127 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:45.366129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:45.366456 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:45.366660 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:45.366661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:45.366661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:45.366800 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:45.367036 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:45.367042 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:45.367202 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 12:56:45.367325 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 12:56:45.367498 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":72,"threads_started":1}
I20260806 12:56:45.368247 27594 heartbeater.cc:347] Connected to a master server at 127.24.83.62:37407
I20260806 12:56:45.368346 27594 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:45.368538 27594 heartbeater.cc:511] Master 127.24.83.62:37407 requested a full tablet report, sending...
I20260806 12:56:45.369086 27395 ts_manager.cc:194] Registered new tserver with Master: 9ea4529583f640d2b761b64dc6dcdd9f (127.24.83.1:37557)
I20260806 12:56:45.369699 27395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:44625
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
W20260806 12:56:46.335495 24908 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.24.83.2: real 0.971s user 0.001s sys 0.002s
I20260806 12:56:46.335873 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:37407
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:56:46.370824 27594 heartbeater.cc:503] Master 127.24.83.62:37407 was elected leader, sending a full tablet report...
W20260806 12:56:46.423202 27599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:46.423393 27599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:46.423416 27599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:46.424666 27599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:46.424710 27599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:46.424737 27599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:56:46.426179 27599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:37407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:46.426425 27599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:46.426672 27599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:46.431701 27607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:46.431710 27605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:46.431716 27604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:46.431823 27599 server_base.cc:1034] running on GCE node
I20260806 12:56:46.432196 27599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:46.432466 27599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:46.433631 27599 hybrid_clock.cc:648] HybridClock initialized: now 1786021006433625 us; error 34 us; skew 500 ppm
Aug 06 12:56:46 dist-test-slave-zkpd krb5kdc[27348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021006, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:46.436656 27599 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:56:46.437121 27599 webserver.cc:460] Webserver started at http://127.24.83.2:35217/ using document root <none> and password file <none>
I20260806 12:56:46.437376 27599 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:46.437423 27599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:46.437595 27599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:46.438663 27599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "a5fe30d235744cc0a1ee59c7e804b369"
format_stamp: "Formatted at 2026-08-06 12:56:46 on dist-test-slave-zkpd"
server_key: "ed9f9a84b41d61e22916449d40e74181"
server_key_iv: "89eb9bfda21be1da32e301b085170d41"
server_key_version: "encryptionkey@0"
I20260806 12:56:46.439064 27599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "a5fe30d235744cc0a1ee59c7e804b369"
format_stamp: "Formatted at 2026-08-06 12:56:46 on dist-test-slave-zkpd"
server_key: "ed9f9a84b41d61e22916449d40e74181"
server_key_iv: "89eb9bfda21be1da32e301b085170d41"
server_key_version: "encryptionkey@0"
I20260806 12:56:46.440672 27599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:56:46.441483 27614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:46.441663 27599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:46.441763 27599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "a5fe30d235744cc0a1ee59c7e804b369"
format_stamp: "Formatted at 2026-08-06 12:56:46 on dist-test-slave-zkpd"
server_key: "ed9f9a84b41d61e22916449d40e74181"
server_key_iv: "89eb9bfda21be1da32e301b085170d41"
server_key_version: "encryptionkey@0"
I20260806 12:56:46.441850 27599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:46.455245 27599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:46.457254 27599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:46.457420 27599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:46.457684 27599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:46.458003 27599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:46.458058 27599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:46.458099 27599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:46.458123 27599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:46.464336 27599 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:43649
I20260806 12:56:46.464375 27730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:43649 every 8 connection(s)
I20260806 12:56:46.464710 27599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:56:46 dist-test-slave-zkpd krb5kdc[27348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021006, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:46.471329 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 27599
I20260806 12:56:46.471439 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:56:46.471668 24908 external_mini_cluster.cc:1428] Setting key c7b5b0ae9e374bc8033c6eb76acd6bab
I20260806 12:56:46.474983 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:46.465478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:60751 (local address 127.24.83.62:37407)
0806 12:56:46.465616 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:46.465618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:46.465705 (+ 87us) server_negotiation.cc:407] Connection header received
0806 12:56:46.466192 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:46.466193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:46.466206 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:46.466271 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:46.467215 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:46.467715 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:46.468727 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:46.468928 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:46.468978 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:46.471085 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:46.471087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:46.471087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:46.471094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:46.473153 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:46.473677 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:46.473689 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:46.473689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:46.473759 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:46.474097 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:46.474099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:46.474099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:46.474262 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:46.474345 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:46.474713 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 12:56:46.474816 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260806 12:56:46.475132 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:46.465422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:37407 (local address 127.24.83.2:60751)
0806 12:56:46.465585 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:46.465629 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:46.466105 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:46.466327 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:46.466330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:46.466940 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:46.467136 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:46.467142 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:46.467795 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:46.467796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:46.468648 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:46.468651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:46.468999 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:46.469000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:46.469097 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:46.469532 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:46.469540 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:46.470984 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:46.473244 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:46.473251 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:46.473253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:46.473594 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:46.473833 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:46.473834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:46.473834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:46.473980 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:46.474354 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:46.474362 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:46.474548 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 12:56:46.474677 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 12:56:46.474863 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260806 12:56:46.475755 27731 heartbeater.cc:347] Connected to a master server at 127.24.83.62:37407
I20260806 12:56:46.475879 27731 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:46.476085 27731 heartbeater.cc:511] Master 127.24.83.62:37407 requested a full tablet report, sending...
I20260806 12:56:46.476473 27395 ts_manager.cc:194] Registered new tserver with Master: a5fe30d235744cc0a1ee59c7e804b369 (127.24.83.2:43649)
I20260806 12:56:46.476869 27395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:60751
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
W20260806 12:56:47.149928 24908 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.24.83.3: real 0.677s user 0.003s sys 0.000s
I20260806 12:56:47.150296 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:37407
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:47.232141 27737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:47.232328 27737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:47.232352 27737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:47.233583 27737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:47.233651 27737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:47.233716 27737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:56:47.235284 27737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:37407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:47.235526 27737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:47.235750 27737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:56:47.240757 27737 server_base.cc:1034] running on GCE node
W20260806 12:56:47.240727 27742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:47.240723 27745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:47.240726 27743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:47.241178 27737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:47.241428 27737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:47.242614 27737 hybrid_clock.cc:648] HybridClock initialized: now 1786021007242594 us; error 31 us; skew 500 ppm
Aug 06 12:56:47 dist-test-slave-zkpd krb5kdc[27348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021007, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:47.245272 27737 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:56:47.245699 27737 webserver.cc:460] Webserver started at http://127.24.83.3:44691/ using document root <none> and password file <none>
I20260806 12:56:47.245934 27737 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:47.246003 27737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:47.246140 27737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:47.247157 27737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "14977a0e284840c1bec313b0cd72f0f4"
format_stamp: "Formatted at 2026-08-06 12:56:47 on dist-test-slave-zkpd"
server_key: "992edc28e3bbebe0e3cd41c3a32d0f1a"
server_key_iv: "a6eb71421e83645c4d4dcdca4cdd9b73"
server_key_version: "encryptionkey@0"
I20260806 12:56:47.247531 27737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "14977a0e284840c1bec313b0cd72f0f4"
format_stamp: "Formatted at 2026-08-06 12:56:47 on dist-test-slave-zkpd"
server_key: "992edc28e3bbebe0e3cd41c3a32d0f1a"
server_key_iv: "a6eb71421e83645c4d4dcdca4cdd9b73"
server_key_version: "encryptionkey@0"
I20260806 12:56:47.248946 27737 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:56:47.249750 27752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:47.249927 27737 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:56:47.250029 27737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "14977a0e284840c1bec313b0cd72f0f4"
format_stamp: "Formatted at 2026-08-06 12:56:47 on dist-test-slave-zkpd"
server_key: "992edc28e3bbebe0e3cd41c3a32d0f1a"
server_key_iv: "a6eb71421e83645c4d4dcdca4cdd9b73"
server_key_version: "encryptionkey@0"
I20260806 12:56:47.250119 27737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:47.275615 27737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:47.278067 27737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:47.278272 27737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:47.278510 27737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:47.278858 27737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:47.278893 27737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:47.278916 27737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:47.278963 27737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:47.284763 27737 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:45023
I20260806 12:56:47.284852 27868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:45023 every 8 connection(s)
I20260806 12:56:47.285174 27737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
Aug 06 12:56:47 dist-test-slave-zkpd krb5kdc[27348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021007, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:47.294471 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:47.285912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:33513 (local address 127.24.83.62:37407)
0806 12:56:47.286138 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:47.286140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:47.286151 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:56:47.286778 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:47.286780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:47.286792 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:47.286864 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:47.287732 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:47.288320 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:47.289327 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:47.289525 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:47.289585 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:47.291695 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:47.291697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:47.291697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:47.291705 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:47.293245 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:47.293624 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:47.293625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:47.293625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:47.293677 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:47.293933 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:47.293933 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:47.293933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:47.294042 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:47.294092 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:47.294327 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 12:56:47.294394 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":58,"threads_started":1}
I20260806 12:56:47.294625 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:47.285864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:37407 (local address 127.24.83.3:33513)
0806 12:56:47.286030 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:47.286064 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:47.286570 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:47.286916 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:47.286919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:47.287464 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:47.287651 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:47.287655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:47.288404 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:47.288405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:47.289252 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:47.289254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:47.289602 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:47.289603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:47.289707 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:47.290163 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:47.290171 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:47.291603 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:47.293331 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:47.293337 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:47.293339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:47.293557 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:47.293748 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:47.293749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:47.293749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:47.293863 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:47.294099 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:47.294104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:47.294218 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 12:56:47.294301 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 12:56:47.294423 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260806 12:56:47.295104 27869 heartbeater.cc:347] Connected to a master server at 127.24.83.62:37407
I20260806 12:56:47.295217 27869 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:47.295432 27869 heartbeater.cc:511] Master 127.24.83.62:37407 requested a full tablet report, sending...
I20260806 12:56:47.295432 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 27737
I20260806 12:56:47.295528 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:56:47.295773 24908 external_mini_cluster.cc:1428] Setting key b304f602c991c1cac9e76be989072530
I20260806 12:56:47.295842 27395 ts_manager.cc:194] Registered new tserver with Master: 14977a0e284840c1bec313b0cd72f0f4 (127.24.83.3:45023)
I20260806 12:56:47.296411 27395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:33513
I20260806 12:56:47.296782 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 12:56:47.308871 27872 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:56:47.307961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60888 (local address 127.24.83.62:37407)
0806 12:56:47.308037 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:47.308039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:47.308138 (+ 99us) server_negotiation.cc:407] Connection header received
0806 12:56:47.308191 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:47.308192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:47.308201 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:47.308252 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:47.308802 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60888: BlockingRecv error: recv got EOF from 127.0.0.1:60888 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:56:47.309684 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 27460
I20260806 12:56:47.314863 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 27599
I20260806 12:56:47.321120 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 27737
I20260806 12:56:47.325661 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 27362
2026-08-06T12:56:47Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3456 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:56:47 dist-test-slave-zkpd krb5kdc[27882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:56:47 dist-test-slave-zkpd krb5kdc[27882](info): set up 2 sockets
Aug 06 12:56:47 dist-test-slave-zkpd krb5kdc[27882](info): commencing operation
krb5kdc: starting...
W20260806 12:56:48.384891 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:56:48 dist-test-slave-zkpd krb5kdc[27882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:56:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:56:48.498690 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:34979
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:41839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:34979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:48.579700 27895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:48.579954 27895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:48.579994 27895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:48.581149 27895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:48.581218 27895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:48.581241 27895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:48.581270 27895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:48.581308 27895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:48.582799 27895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:41839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:34979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:34979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:48.583079 27895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:48.583312 27895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:48.588083 27901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:48.588088 27903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:48.588088 27900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:48.588311 27895 server_base.cc:1034] running on GCE node
I20260806 12:56:48.588625 27895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:48.588914 27895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:48.590080 27895 hybrid_clock.cc:648] HybridClock initialized: now 1786021008590066 us; error 47 us; skew 500 ppm
Aug 06 12:56:48 dist-test-slave-zkpd krb5kdc[27882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021008, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:48.592612 27895 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:56:48.593000 27895 webserver.cc:460] Webserver started at http://127.24.83.62:43227/ using document root <none> and password file <none>
I20260806 12:56:48.593223 27895 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:48.593267 27895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:48.593405 27895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:48.594482 27895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "c6275c17aacc49e58ef9b29f9ba9fc27"
format_stamp: "Formatted at 2026-08-06 12:56:48 on dist-test-slave-zkpd"
server_key: "86032b08f80dc1870929844723916a95"
server_key_iv: "e6e781aaf239640347ede65b1db608bf"
server_key_version: "encryptionkey@0"
I20260806 12:56:48.594838 27895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "c6275c17aacc49e58ef9b29f9ba9fc27"
format_stamp: "Formatted at 2026-08-06 12:56:48 on dist-test-slave-zkpd"
server_key: "86032b08f80dc1870929844723916a95"
server_key_iv: "e6e781aaf239640347ede65b1db608bf"
server_key_version: "encryptionkey@0"
I20260806 12:56:48.596232 27895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 12:56:48.597002 27911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:48.597201 27895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:48.597276 27895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "c6275c17aacc49e58ef9b29f9ba9fc27"
format_stamp: "Formatted at 2026-08-06 12:56:48 on dist-test-slave-zkpd"
server_key: "86032b08f80dc1870929844723916a95"
server_key_iv: "e6e781aaf239640347ede65b1db608bf"
server_key_version: "encryptionkey@0"
I20260806 12:56:48.597364 27895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:48.612941 27895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:48.614715 27895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:48.614871 27895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:48.619068 27895 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:34979
I20260806 12:56:48.619436 27895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:48.619457 27963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:34979 every 8 connection(s)
I20260806 12:56:48.620203 27965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:48.622470 27965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27: Bootstrap starting.
I20260806 12:56:48.623046 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 27895
I20260806 12:56:48.623149 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:48.623123 27965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:48.623323 24908 external_mini_cluster.cc:1428] Setting key ac290122d227ebad2303ae6d09bb40bf
I20260806 12:56:48.623486 27965 log.cc:824] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:48.624296 27965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27: No bootstrap required, opened a new log
I20260806 12:56:48.626619 27965 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6275c17aacc49e58ef9b29f9ba9fc27" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34979 } }
I20260806 12:56:48.626793 27965 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:48.626832 27965 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6275c17aacc49e58ef9b29f9ba9fc27, State: Initialized, Role: FOLLOWER
I20260806 12:56:48.626960 27965 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6275c17aacc49e58ef9b29f9ba9fc27" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34979 } }
I20260806 12:56:48.627030 27965 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:48.627059 27965 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:48.627101 27965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:48.627718 27965 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6275c17aacc49e58ef9b29f9ba9fc27" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34979 } }
I20260806 12:56:48.627849 27965 leader_election.cc:302] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6275c17aacc49e58ef9b29f9ba9fc27; no voters:
I20260806 12:56:48.628069 27965 leader_election.cc:288] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:48.628098 27970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:48.628335 27970 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [term 1 LEADER]: Becoming Leader. State: Replica: c6275c17aacc49e58ef9b29f9ba9fc27, State: Running, Role: LEADER
I20260806 12:56:48.628448 27965 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:48.628432 27970 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6275c17aacc49e58ef9b29f9ba9fc27" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34979 } }
Aug 06 12:56:48 dist-test-slave-zkpd krb5kdc[27882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:48.630535 27972 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6275c17aacc49e58ef9b29f9ba9fc27. Latest consensus state: current_term: 1 leader_uuid: "c6275c17aacc49e58ef9b29f9ba9fc27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6275c17aacc49e58ef9b29f9ba9fc27" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34979 } } }
I20260806 12:56:48.630652 27972 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [sys.catalog]: This master's current role is: LEADER
W20260806 12:56:48.630769 27984 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:56:48.630839 27984 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:56:48.630336 27971 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6275c17aacc49e58ef9b29f9ba9fc27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6275c17aacc49e58ef9b29f9ba9fc27" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34979 } } }
I20260806 12:56:48.630990 27971 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:48.630990 27987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:48.631281 27987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:48.632969 27987 catalog_manager.cc:1347] Generated new cluster ID: 709506c1f8a444699869b7229382dcd1
I20260806 12:56:48.633033 27987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:56:48.634389 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:48.623822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55910 (local address 127.24.83.62:34979)
0806 12:56:48.624011 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:48.624020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:48.624056 (+ 36us) server_negotiation.cc:407] Connection header received
0806 12:56:48.624530 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:48.624537 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:48.624583 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:48.624696 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:48.625505 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:48.626321 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:48.627375 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:48.627768 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:48.627953 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:48.630687 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:48.630692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:48.630692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:48.630704 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:48.632700 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:48.633142 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:48.633145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:48.633145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:48.633247 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:48.633479 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:48.633479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:48.633480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:48.633769 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:48.633896 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:48.634056 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 12:56:48.634185 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260806 12:56:48.635255 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:56:48.643879 27987 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:48.644366 27987 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:48.648268 27987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6275c17aacc49e58ef9b29f9ba9fc27: Generated new TSK 0
I20260806 12:56:48.648437 27987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:56:48.660261 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:48.656797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.24.83.62:34979)
0806 12:56:48.656838 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:48.656840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:48.656976 (+ 136us) server_negotiation.cc:407] Connection header received
0806 12:56:48.657061 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:48.657062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:48.657072 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:48.657141 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:48.657511 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:48.658030 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:48.658767 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:48.658991 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:48.659065 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:48.659758 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:48.659760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:48.659760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:48.659996 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:48.660012 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:48.660018 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:48.660084 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:48.660151 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 12:56:48.660178 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 12:56:48.662710 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 27895
2026-08-06T12:56:48Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1335 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:56:48 dist-test-slave-zkpd krb5kdc[27998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:56:48 dist-test-slave-zkpd krb5kdc[27998](info): set up 2 sockets
Aug 06 12:56:48 dist-test-slave-zkpd krb5kdc[27998](info): commencing operation
krb5kdc: starting...
W20260806 12:56:49.699613 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:56:49 dist-test-slave-zkpd krb5kdc[27998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:56:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:56:49.812377 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:43773
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:43773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:49.892017 28012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:49.892199 28012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:49.892220 28012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:49.893253 28012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:49.893286 28012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:49.893301 28012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:49.893312 28012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:49.893324 28012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:49.894682 28012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:43773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:43773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:49.894862 28012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:49.895032 28012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:49.899742 28018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:49.899811 28017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:49.899856 28012 server_base.cc:1034] running on GCE node
W20260806 12:56:49.899744 28020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:49.900189 28012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:49.900473 28012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:49.901643 28012 hybrid_clock.cc:648] HybridClock initialized: now 1786021009901612 us; error 31 us; skew 500 ppm
Aug 06 12:56:49 dist-test-slave-zkpd krb5kdc[27998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021009, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:49.907703 28012 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:56:49.908114 28012 webserver.cc:460] Webserver started at http://127.24.83.62:37727/ using document root <none> and password file <none>
I20260806 12:56:49.908283 28012 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:49.908322 28012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:49.908413 28012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:49.909375 28012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "98ba87bfdeab4171b98ab5b1cb7028a7"
format_stamp: "Formatted at 2026-08-06 12:56:49 on dist-test-slave-zkpd"
server_key: "94749af41556aa28dc2217dc1890247f"
server_key_iv: "9e2b13700c61c8142bbb15dfbc5dda3c"
server_key_version: "encryptionkey@0"
I20260806 12:56:49.909684 28012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "98ba87bfdeab4171b98ab5b1cb7028a7"
format_stamp: "Formatted at 2026-08-06 12:56:49 on dist-test-slave-zkpd"
server_key: "94749af41556aa28dc2217dc1890247f"
server_key_iv: "9e2b13700c61c8142bbb15dfbc5dda3c"
server_key_version: "encryptionkey@0"
I20260806 12:56:49.910846 28012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:49.911535 28027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:49.911707 28012 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:49.911791 28012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "98ba87bfdeab4171b98ab5b1cb7028a7"
format_stamp: "Formatted at 2026-08-06 12:56:49 on dist-test-slave-zkpd"
server_key: "94749af41556aa28dc2217dc1890247f"
server_key_iv: "9e2b13700c61c8142bbb15dfbc5dda3c"
server_key_version: "encryptionkey@0"
I20260806 12:56:49.911844 28012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:49.927829 28012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:49.929631 28012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:49.929798 28012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:49.933460 28012 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:43773
I20260806 12:56:49.933561 28080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:43773 every 8 connection(s)
I20260806 12:56:49.933868 28012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:49.934744 28081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:49.936906 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 28012
I20260806 12:56:49.937012 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:49.937211 24908 external_mini_cluster.cc:1428] Setting key be5eb0de3f7c8002f6083df632ba0e55
I20260806 12:56:49.937278 28081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7: Bootstrap starting.
I20260806 12:56:49.937922 28081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:49.938203 28081 log.cc:824] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:49.938855 28081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7: No bootstrap required, opened a new log
I20260806 12:56:49.940665 28081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ba87bfdeab4171b98ab5b1cb7028a7" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43773 } }
I20260806 12:56:49.940816 28081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:49.940881 28081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ba87bfdeab4171b98ab5b1cb7028a7, State: Initialized, Role: FOLLOWER
I20260806 12:56:49.941020 28081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ba87bfdeab4171b98ab5b1cb7028a7" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43773 } }
I20260806 12:56:49.941124 28081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:49.941169 28081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:49.941216 28081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:49.941751 28081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ba87bfdeab4171b98ab5b1cb7028a7" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43773 } }
I20260806 12:56:49.941865 28081 leader_election.cc:302] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98ba87bfdeab4171b98ab5b1cb7028a7; no voters:
I20260806 12:56:49.942036 28081 leader_election.cc:288] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:49.942065 28086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:49.942257 28086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [term 1 LEADER]: Becoming Leader. State: Replica: 98ba87bfdeab4171b98ab5b1cb7028a7, State: Running, Role: LEADER
I20260806 12:56:49.942283 28081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:49.942426 28086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ba87bfdeab4171b98ab5b1cb7028a7" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43773 } }
I20260806 12:56:49.943468 28086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98ba87bfdeab4171b98ab5b1cb7028a7. Latest consensus state: current_term: 1 leader_uuid: "98ba87bfdeab4171b98ab5b1cb7028a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ba87bfdeab4171b98ab5b1cb7028a7" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43773 } } }
I20260806 12:56:49.943588 28086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:49.943476 28088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98ba87bfdeab4171b98ab5b1cb7028a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ba87bfdeab4171b98ab5b1cb7028a7" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43773 } } }
I20260806 12:56:49.943800 28095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:49.943856 28088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7 [sys.catalog]: This master's current role is: LEADER
Aug 06 12:56:49 dist-test-slave-zkpd krb5kdc[27998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:49.944430 28095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:49.946163 28095 catalog_manager.cc:1347] Generated new cluster ID: 51f0fe70f52b408993f5dc250a65c8fa
I20260806 12:56:49.946223 28095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:56:49.948302 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:49.937718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56810 (local address 127.24.83.62:43773)
0806 12:56:49.937916 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:49.937925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:49.937980 (+ 55us) server_negotiation.cc:407] Connection header received
0806 12:56:49.938487 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:49.938494 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:49.938540 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:49.938680 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:49.939402 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:49.940334 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:49.941234 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:49.941674 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:49.941795 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:49.944666 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:49.944670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:49.944670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:49.944681 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:49.946686 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:49.947074 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:49.947077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:49.947077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:49.947185 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:49.947392 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:49.947393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:49.947393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:49.947684 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:49.947829 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:49.947958 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 12:56:49.948094 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":76,"threads_started":1}
I20260806 12:56:49.958976 28095 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:49.959478 28095 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:49.965986 28095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98ba87bfdeab4171b98ab5b1cb7028a7: Generated new TSK 0
I20260806 12:56:49.966146 28095 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:56:49.992991 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43773
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:50.073647 28107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:50.073896 28107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:50.073935 28107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:50.075083 28107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:50.075150 28107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:50.075191 28107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:56:50.076659 28107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:50.076920 28107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:50.077176 28107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:56:50.083420 28107 server_base.cc:1034] running on GCE node
W20260806 12:56:50.083431 28113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:50.083534 28112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:50.083549 28115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:50.083755 28107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:50.084055 28107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:50.085207 28107 hybrid_clock.cc:648] HybridClock initialized: now 1786021010085188 us; error 39 us; skew 500 ppm
Aug 06 12:56:50 dist-test-slave-zkpd krb5kdc[27998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021010, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:50.087850 28107 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:56:50.088316 28107 webserver.cc:460] Webserver started at http://127.24.83.1:38655/ using document root <none> and password file <none>
I20260806 12:56:50.088552 28107 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:50.088598 28107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:50.088744 28107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:50.089795 28107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "838deb209dc54deb8ef23815ac1274f9"
format_stamp: "Formatted at 2026-08-06 12:56:50 on dist-test-slave-zkpd"
server_key: "5f2ccc90ebc0f593e8b950ccfa01f066"
server_key_iv: "27ceceff4c8a5603ff23cb16afdabf51"
server_key_version: "encryptionkey@0"
I20260806 12:56:50.090163 28107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "838deb209dc54deb8ef23815ac1274f9"
format_stamp: "Formatted at 2026-08-06 12:56:50 on dist-test-slave-zkpd"
server_key: "5f2ccc90ebc0f593e8b950ccfa01f066"
server_key_iv: "27ceceff4c8a5603ff23cb16afdabf51"
server_key_version: "encryptionkey@0"
I20260806 12:56:50.091476 28107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 12:56:50.092286 28122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:50.092458 28107 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:50.092566 28107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "838deb209dc54deb8ef23815ac1274f9"
format_stamp: "Formatted at 2026-08-06 12:56:50 on dist-test-slave-zkpd"
server_key: "5f2ccc90ebc0f593e8b950ccfa01f066"
server_key_iv: "27ceceff4c8a5603ff23cb16afdabf51"
server_key_version: "encryptionkey@0"
I20260806 12:56:50.092643 28107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:50.105799 28107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:50.107540 28107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:50.107704 28107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:50.107978 28107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:50.108304 28107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:50.108361 28107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:50.108394 28107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:50.108422 28107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:50.114130 28107 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:40853
I20260806 12:56:50.114203 28240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:40853 every 8 connection(s)
I20260806 12:56:50.114564 28107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:56:50.117939 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 28107
I20260806 12:56:50.118019 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:56:50.118217 24908 external_mini_cluster.cc:1428] Setting key 7506e6bac1eadfb9c2937ae6d02bda4c
Aug 06 12:56:50 dist-test-slave-zkpd krb5kdc[27998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021010, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:50.124406 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:50.115273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:49139 (local address 127.24.83.62:43773)
0806 12:56:50.115343 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:50.115345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:50.115584 (+ 239us) server_negotiation.cc:407] Connection header received
0806 12:56:50.116131 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:50.116132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:50.116142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:50.116188 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:50.117080 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.117581 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.118849 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.119065 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.119149 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:50.121356 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:50.121358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:50.121358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:50.121366 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:50.122886 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:50.123299 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:50.123300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:50.123301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:50.123364 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:50.123627 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:50.123627 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:50.123628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:50.123753 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:50.123829 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:50.124189 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 12:56:50.124303 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 12:56:50.124552 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:50.115269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:43773 (local address 127.24.83.1:49139)
0806 12:56:50.115451 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:50.115496 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:50.116040 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:50.116248 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:50.116251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:50.116819 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:50.117003 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:50.117008 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.117661 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.117661 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:50.118763 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:50.118767 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.119158 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.119160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:50.119303 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:50.119831 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:50.119839 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:50.121254 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:50.122968 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:50.122975 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:50.122978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:50.123208 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:50.123432 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:50.123433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:50.123433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:50.123548 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:50.123831 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:50.123839 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:50.124048 (+ 209us) client_negotiation.cc:769] Sending connection context
0806 12:56:50.124154 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 12:56:50.124324 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":89,"threads_started":1}
I20260806 12:56:50.125057 28241 heartbeater.cc:347] Connected to a master server at 127.24.83.62:43773
I20260806 12:56:50.125171 28241 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:50.125379 28241 heartbeater.cc:511] Master 127.24.83.62:43773 requested a full tablet report, sending...
I20260806 12:56:50.125929 28045 ts_manager.cc:194] Registered new tserver with Master: 838deb209dc54deb8ef23815ac1274f9 (127.24.83.1:40853)
I20260806 12:56:50.126549 28045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:49139
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:56:50.142681 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43773
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:50.225116 28247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:50.225436 28247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:50.225478 28247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:50.226634 28247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:50.226714 28247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:50.226763 28247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:56:50.228226 28247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:50.228513 28247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:50.228768 28247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:50.233819 28253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:50.233836 28255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:50.233940 28247 server_base.cc:1034] running on GCE node
W20260806 12:56:50.233858 28252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:50.234149 28247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:50.234386 28247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:50.235531 28247 hybrid_clock.cc:648] HybridClock initialized: now 1786021010235506 us; error 43 us; skew 500 ppm
Aug 06 12:56:50 dist-test-slave-zkpd krb5kdc[27998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021010, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:50.237994 28247 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:56:50.238374 28247 webserver.cc:460] Webserver started at http://127.24.83.2:34627/ using document root <none> and password file <none>
I20260806 12:56:50.238569 28247 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:50.238608 28247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:50.238694 28247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:50.239661 28247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "435c7935176643f0a09e6abac7060022"
format_stamp: "Formatted at 2026-08-06 12:56:50 on dist-test-slave-zkpd"
server_key: "55028910f30aef5f740c8d3a993bc8ef"
server_key_iv: "5d824ca9d127eb2fd1a2174ec537fdb9"
server_key_version: "encryptionkey@0"
I20260806 12:56:50.240023 28247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "435c7935176643f0a09e6abac7060022"
format_stamp: "Formatted at 2026-08-06 12:56:50 on dist-test-slave-zkpd"
server_key: "55028910f30aef5f740c8d3a993bc8ef"
server_key_iv: "5d824ca9d127eb2fd1a2174ec537fdb9"
server_key_version: "encryptionkey@0"
I20260806 12:56:50.241261 28247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:50.241919 28262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:50.242093 28247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:50.242208 28247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "435c7935176643f0a09e6abac7060022"
format_stamp: "Formatted at 2026-08-06 12:56:50 on dist-test-slave-zkpd"
server_key: "55028910f30aef5f740c8d3a993bc8ef"
server_key_iv: "5d824ca9d127eb2fd1a2174ec537fdb9"
server_key_version: "encryptionkey@0"
I20260806 12:56:50.242297 28247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:50.253166 28247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:50.254870 28247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:50.255020 28247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:50.255229 28247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:50.255525 28247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:50.255577 28247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:50.255616 28247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:50.255641 28247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:50.261428 28247 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:35335
I20260806 12:56:50.261478 28379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:35335 every 8 connection(s)
I20260806 12:56:50.261758 28247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:56:50.267305 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 28247
I20260806 12:56:50.267422 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:56:50.267608 24908 external_mini_cluster.cc:1428] Setting key 7f28a33ad920c5755e26a710b311e2c5
Aug 06 12:56:50 dist-test-slave-zkpd krb5kdc[27998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021010, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:50.271039 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:50.262430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:52377 (local address 127.24.83.62:43773)
0806 12:56:50.262498 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:50.262500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:50.262671 (+ 171us) server_negotiation.cc:407] Connection header received
0806 12:56:50.263113 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:50.263114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:50.263123 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:50.263177 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:50.264040 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.264527 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.265441 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.265610 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.265657 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:50.268221 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:50.268223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:50.268223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:50.268230 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:50.269825 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:50.270203 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:50.270204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:50.270205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:50.270269 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:50.270473 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:50.270473 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:50.270474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:50.270620 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:50.270697 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:50.270914 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 12:56:50.270975 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:56:50.271162 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:50.262398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:43773 (local address 127.24.83.2:52377)
0806 12:56:50.262546 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:50.262584 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:50.263030 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:50.263226 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:50.263229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:50.263761 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:50.263964 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:50.263969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.264608 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.264608 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:50.265369 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:50.265370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.265679 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.265680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:50.265786 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:50.266330 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:50.266338 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:50.268119 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:50.269917 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:50.269923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:50.269924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:50.270129 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:50.270333 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:50.270333 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:50.270333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:50.270413 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:50.270684 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:50.270687 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:50.270797 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 12:56:50.270873 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 12:56:50.270981 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260806 12:56:50.271588 28380 heartbeater.cc:347] Connected to a master server at 127.24.83.62:43773
I20260806 12:56:50.271677 28380 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:50.271860 28380 heartbeater.cc:511] Master 127.24.83.62:43773 requested a full tablet report, sending...
I20260806 12:56:50.272289 28045 ts_manager.cc:194] Registered new tserver with Master: 435c7935176643f0a09e6abac7060022 (127.24.83.2:35335)
I20260806 12:56:50.272645 28045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:52377
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:56:50.295596 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43773
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:50.378525 28385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:50.378726 28385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:50.378749 28385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:50.379927 28385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:50.379977 28385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:50.380016 28385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:56:50.381481 28385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:50.381664 28385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:50.381841 28385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:50.386620 28390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:50.386656 28391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:50.386637 28393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:50.386999 28385 server_base.cc:1034] running on GCE node
I20260806 12:56:50.387197 28385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:50.387422 28385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:50.388604 28385 hybrid_clock.cc:648] HybridClock initialized: now 1786021010388602 us; error 38 us; skew 500 ppm
Aug 06 12:56:50 dist-test-slave-zkpd krb5kdc[27998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021010, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:50.391011 28385 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:56:50.391497 28385 webserver.cc:460] Webserver started at http://127.24.83.3:44713/ using document root <none> and password file <none>
I20260806 12:56:50.391719 28385 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:50.391795 28385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:50.391968 28385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:50.393004 28385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "ed898ac542d6494dab0edad8698e624b"
format_stamp: "Formatted at 2026-08-06 12:56:50 on dist-test-slave-zkpd"
server_key: "f3de4f6fa7b3b6016b866018c19fcd77"
server_key_iv: "cb9b7b325e7cf2c7ceacc36f5ad8cee0"
server_key_version: "encryptionkey@0"
I20260806 12:56:50.393385 28385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "ed898ac542d6494dab0edad8698e624b"
format_stamp: "Formatted at 2026-08-06 12:56:50 on dist-test-slave-zkpd"
server_key: "f3de4f6fa7b3b6016b866018c19fcd77"
server_key_iv: "cb9b7b325e7cf2c7ceacc36f5ad8cee0"
server_key_version: "encryptionkey@0"
I20260806 12:56:50.395174 28385 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:56:50.395999 28400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:50.396183 28385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:50.396282 28385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "ed898ac542d6494dab0edad8698e624b"
format_stamp: "Formatted at 2026-08-06 12:56:50 on dist-test-slave-zkpd"
server_key: "f3de4f6fa7b3b6016b866018c19fcd77"
server_key_iv: "cb9b7b325e7cf2c7ceacc36f5ad8cee0"
server_key_version: "encryptionkey@0"
I20260806 12:56:50.396373 28385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:50.405305 28385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:50.407075 28385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:50.407230 28385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:50.407456 28385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:50.407765 28385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:50.407825 28385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:50.407866 28385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:50.407939 28385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:50.414001 28385 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:38011
I20260806 12:56:50.414067 28516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:38011 every 8 connection(s)
I20260806 12:56:50.414389 28385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
Aug 06 12:56:50 dist-test-slave-zkpd krb5kdc[27998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021010, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:50.420607 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 28385
I20260806 12:56:50.420719 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:56:50.420928 24908 external_mini_cluster.cc:1428] Setting key d9f465458d999c2b41ac4a32ebb5e75d
I20260806 12:56:50.423525 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:50.415090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:58375 (local address 127.24.83.62:43773)
0806 12:56:50.415159 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:50.415161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:50.415390 (+ 229us) server_negotiation.cc:407] Connection header received
0806 12:56:50.415921 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:50.415922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:50.415934 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:50.415988 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:50.416930 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.417445 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.418440 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.418644 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.418729 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:50.420847 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:50.420849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:50.420849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:50.420857 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:50.422414 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:50.422764 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:50.422764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:50.422765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:50.422818 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:50.423035 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:50.423035 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:50.423035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:50.423137 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:50.423185 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:50.423404 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 12:56:50.423463 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:56:50.423683 28519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:50.415070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:43773 (local address 127.24.83.3:58375)
0806 12:56:50.415267 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:50.415302 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:50.415795 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:50.416064 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:50.416068 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:50.416628 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:50.416809 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:50.416815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.417541 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.417542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:50.418341 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:50.418344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.418740 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.418740 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:50.418867 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:50.419375 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:50.419383 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:50.420734 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:50.422483 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:50.422489 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:50.422490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:50.422697 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:50.422877 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:50.422877 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:50.422877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:50.422979 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:50.423191 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:50.423195 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:50.423299 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 12:56:50.423376 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 12:56:50.423490 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260806 12:56:50.424119 28517 heartbeater.cc:347] Connected to a master server at 127.24.83.62:43773
I20260806 12:56:50.424220 28517 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:50.424386 28517 heartbeater.cc:511] Master 127.24.83.62:43773 requested a full tablet report, sending...
I20260806 12:56:50.424767 28045 ts_manager.cc:194] Registered new tserver with Master: ed898ac542d6494dab0edad8698e624b (127.24.83.3:38011)
I20260806 12:56:50.425143 28045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:58375
I20260806 12:56:50.432622 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 12:56:50 dist-test-slave-zkpd krb5kdc[27998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021010, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 12:56:50 dist-test-slave-zkpd krb5kdc[27998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021010, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:50.454540 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:50.448206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.24.83.62:43773)
0806 12:56:50.448272 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:50.448274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:50.448418 (+ 144us) server_negotiation.cc:407] Connection header received
0806 12:56:50.448464 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:50.448465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:50.448474 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:50.448515 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:50.449025 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.449493 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.450174 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:50.450345 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:50.450398 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:50.452471 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:50.452472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:50.452472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:50.452478 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:50.453625 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:50.453938 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:50.453939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:50.453939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:50.453985 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:50.454145 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:50.454145 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:50.454145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:50.454268 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:50.454339 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:50.454421 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 12:56:50.454477 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 12:56:50.454767 28045 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34882
I20260806 12:56:50.455158 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 28107
I20260806 12:56:50.460737 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 28247
I20260806 12:56:50.465632 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 28385
I20260806 12:56:50.469991 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 28012
2026-08-06T12:56:50Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1809 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:56:50 dist-test-slave-zkpd krb5kdc[28529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:56:50 dist-test-slave-zkpd krb5kdc[28529](info): set up 2 sockets
Aug 06 12:56:50 dist-test-slave-zkpd krb5kdc[28529](info): commencing operation
krb5kdc: starting...
W20260806 12:56:51.519649 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:56:51 dist-test-slave-zkpd krb5kdc[28529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:56:51.634363 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40851
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:33533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:40851
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:51.713860 28542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:51.714092 28542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:51.714128 28542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:51.715250 28542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:51.715317 28542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:51.715340 28542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:51.715376 28542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:51.715406 28542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:51.716903 28542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:33533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:40851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:51.717170 28542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:51.717394 28542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:51.722096 28551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:51.722126 28549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:51.722127 28547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:51.722357 28542 server_base.cc:1034] running on GCE node
I20260806 12:56:51.722627 28542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:51.722910 28542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:51.724078 28542 hybrid_clock.cc:648] HybridClock initialized: now 1786021011724065 us; error 42 us; skew 500 ppm
Aug 06 12:56:51 dist-test-slave-zkpd krb5kdc[28529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021011, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:51.726418 28542 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:56:51.726878 28542 webserver.cc:460] Webserver started at http://127.24.83.62:44231/ using document root <none> and password file <none>
I20260806 12:56:51.727097 28542 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:51.727162 28542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:51.727293 28542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:51.728354 28542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "1385da9b2605451f97ad7406a99c59c8"
format_stamp: "Formatted at 2026-08-06 12:56:51 on dist-test-slave-zkpd"
server_key: "a6191f0a3428af894ebcf169a7d816ac"
server_key_iv: "3c52ac813a7fe6f3f019ec92200b6f7d"
server_key_version: "encryptionkey@0"
I20260806 12:56:51.728713 28542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "1385da9b2605451f97ad7406a99c59c8"
format_stamp: "Formatted at 2026-08-06 12:56:51 on dist-test-slave-zkpd"
server_key: "a6191f0a3428af894ebcf169a7d816ac"
server_key_iv: "3c52ac813a7fe6f3f019ec92200b6f7d"
server_key_version: "encryptionkey@0"
I20260806 12:56:51.730006 28542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:56:51.730722 28558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:51.730895 28542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:51.730995 28542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "1385da9b2605451f97ad7406a99c59c8"
format_stamp: "Formatted at 2026-08-06 12:56:51 on dist-test-slave-zkpd"
server_key: "a6191f0a3428af894ebcf169a7d816ac"
server_key_iv: "3c52ac813a7fe6f3f019ec92200b6f7d"
server_key_version: "encryptionkey@0"
I20260806 12:56:51.731077 28542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:51.745103 28542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:51.746809 28542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:51.746973 28542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:51.750941 28542 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:40851
I20260806 12:56:51.750974 28610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:40851 every 8 connection(s)
I20260806 12:56:51.751489 28542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:51.752133 28611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:51.754240 28611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8: Bootstrap starting.
I20260806 12:56:51.754756 28611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:51.755009 28611 log.cc:824] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:51.755621 28611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8: No bootstrap required, opened a new log
I20260806 12:56:51.757179 28611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1385da9b2605451f97ad7406a99c59c8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40851 } }
I20260806 12:56:51.757310 28611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:51.757333 28611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1385da9b2605451f97ad7406a99c59c8, State: Initialized, Role: FOLLOWER
I20260806 12:56:51.757460 28611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1385da9b2605451f97ad7406a99c59c8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40851 } }
I20260806 12:56:51.757539 28611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:51.757561 28611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:51.757589 28611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:51.758451 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 28542
I20260806 12:56:51.758515 28611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1385da9b2605451f97ad7406a99c59c8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40851 } }
I20260806 12:56:51.758569 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:51.758627 28611 leader_election.cc:302] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1385da9b2605451f97ad7406a99c59c8; no voters:
I20260806 12:56:51.758771 24908 external_mini_cluster.cc:1428] Setting key 8c3335201e0285a36496db438df23c86
I20260806 12:56:51.758803 28611 leader_election.cc:288] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:51.758862 28614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:51.759058 28611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:51.759056 28614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [term 1 LEADER]: Becoming Leader. State: Replica: 1385da9b2605451f97ad7406a99c59c8, State: Running, Role: LEADER
I20260806 12:56:51.759341 28614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1385da9b2605451f97ad7406a99c59c8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40851 } }
I20260806 12:56:51.759738 28616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1385da9b2605451f97ad7406a99c59c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1385da9b2605451f97ad7406a99c59c8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40851 } } }
I20260806 12:56:51.759795 28617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1385da9b2605451f97ad7406a99c59c8. Latest consensus state: current_term: 1 leader_uuid: "1385da9b2605451f97ad7406a99c59c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1385da9b2605451f97ad7406a99c59c8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40851 } } }
I20260806 12:56:51.759909 28616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:51.760010 28617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:51.760262 28626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:51.760861 28626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:51.762516 28626 catalog_manager.cc:1347] Generated new cluster ID: bd1678bd14c84fa4861f9a1a2459abc0
I20260806 12:56:51.762570 28626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:56:51 dist-test-slave-zkpd krb5kdc[28529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:51.769903 28626 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:51.770129 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:51.759194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.24.83.62:40851)
0806 12:56:51.759513 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:51.759523 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:51.759862 (+ 339us) server_negotiation.cc:407] Connection header received
0806 12:56:51.760122 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:51.760130 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:51.760189 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:51.760302 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:51.761234 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:51.762517 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:51.763328 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:51.763763 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:51.763879 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:51.766129 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:51.766132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:51.766133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:51.766144 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:51.768381 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:51.768740 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:51.768745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:51.768745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:51.768827 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:51.769052 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:51.769053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:51.769053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:51.769274 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:51.769433 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:51.769724 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 12:56:51.769825 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
I20260806 12:56:51.770599 28626 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:51.778355 28626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1385da9b2605451f97ad7406a99c59c8: Generated new TSK 0
I20260806 12:56:51.778532 28626 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:56:51.806872 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40851
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:33533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:51.888094 28638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:51.888329 28638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:51.888370 28638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:51.889528 28638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:51.889595 28638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:51.889642 28638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:56:51.891191 28638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:33533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:51.891425 28638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:51.891614 28638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:51.896591 28643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:51.896598 28644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:51.896598 28646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:51.896950 28638 server_base.cc:1034] running on GCE node
I20260806 12:56:51.897136 28638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:51.897373 28638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:51.898518 28638 hybrid_clock.cc:648] HybridClock initialized: now 1786021011898506 us; error 26 us; skew 500 ppm
Aug 06 12:56:51 dist-test-slave-zkpd krb5kdc[28529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021011, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:51.901031 28638 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:56:51.901463 28638 webserver.cc:460] Webserver started at http://127.24.83.1:43641/ using document root <none> and password file <none>
I20260806 12:56:51.901697 28638 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:51.901742 28638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:51.901906 28638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:51.902927 28638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "bea7d192bd5e4b90a8c6dc2011866f81"
format_stamp: "Formatted at 2026-08-06 12:56:51 on dist-test-slave-zkpd"
server_key: "03b22652fe5aee754a9d63ec6b3e46cd"
server_key_iv: "d674e5c833d7f4b23ec7e448ca35ed29"
server_key_version: "encryptionkey@0"
I20260806 12:56:51.903295 28638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "bea7d192bd5e4b90a8c6dc2011866f81"
format_stamp: "Formatted at 2026-08-06 12:56:51 on dist-test-slave-zkpd"
server_key: "03b22652fe5aee754a9d63ec6b3e46cd"
server_key_iv: "d674e5c833d7f4b23ec7e448ca35ed29"
server_key_version: "encryptionkey@0"
I20260806 12:56:51.904677 28638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:56:51.905400 28653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:51.905584 28638 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:56:51.905678 28638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "bea7d192bd5e4b90a8c6dc2011866f81"
format_stamp: "Formatted at 2026-08-06 12:56:51 on dist-test-slave-zkpd"
server_key: "03b22652fe5aee754a9d63ec6b3e46cd"
server_key_iv: "d674e5c833d7f4b23ec7e448ca35ed29"
server_key_version: "encryptionkey@0"
I20260806 12:56:51.905751 28638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:51.913491 28638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:51.915174 28638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:51.915337 28638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:51.915550 28638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:51.915853 28638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:51.915939 28638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:51.915987 28638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:51.916015 28638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:51.921789 28638 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:43483
I20260806 12:56:51.921867 28769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:43483 every 8 connection(s)
I20260806 12:56:51.922163 28638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
Aug 06 12:56:51 dist-test-slave-zkpd krb5kdc[28529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021011, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:51.930938 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:51.922862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:35921 (local address 127.24.83.62:40851)
0806 12:56:51.922934 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:51.922936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:51.923119 (+ 183us) server_negotiation.cc:407] Connection header received
0806 12:56:51.924105 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:51.924106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:51.924119 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:51.924169 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:51.924719 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:51.925215 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:51.926155 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:51.926348 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:51.926402 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:51.928370 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:51.928372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:51.928372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:51.928380 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:51.929580 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:51.929983 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:51.929984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:51.929985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:51.930047 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:51.930291 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:51.930292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:51.930293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:51.930407 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:51.930478 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:51.930746 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 12:56:51.930839 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:56:51.931084 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:51.922833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40851 (local address 127.24.83.1:35921)
0806 12:56:51.923007 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:51.923041 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:51.924013 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:51.924220 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:51.924223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:51.924464 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:51.924642 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:51.924647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:51.925302 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:51.925303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:51.926076 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:51.926078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:51.926422 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:51.926423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:51.926518 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:51.926934 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:51.926942 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:51.928268 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:51.929675 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:51.929682 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:51.929684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:51.929904 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:51.930115 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:51.930116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:51.930117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:51.930222 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:51.930480 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:51.930488 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:51.930617 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 12:56:51.930723 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 12:56:51.930867 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260806 12:56:51.931630 28770 heartbeater.cc:347] Connected to a master server at 127.24.83.62:40851
I20260806 12:56:51.931707 28770 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:51.931937 28770 heartbeater.cc:511] Master 127.24.83.62:40851 requested a full tablet report, sending...
I20260806 12:56:51.931982 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 28638
I20260806 12:56:51.932101 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:56:51.932276 24908 external_mini_cluster.cc:1428] Setting key 29980c78d470c45f60b749c641146ce7
I20260806 12:56:51.932585 28575 ts_manager.cc:194] Registered new tserver with Master: bea7d192bd5e4b90a8c6dc2011866f81 (127.24.83.1:43483)
I20260806 12:56:51.933490 28575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:35921
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:56:51.956683 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40851
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:33533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:52.036466 28775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:52.036649 28775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:52.036674 28775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:52.037729 28775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:52.037761 28775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:52.037786 28775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:56:52.039203 28775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:33533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:52.039384 28775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:52.039671 28775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:52.044603 28781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:52.044606 28775 server_base.cc:1034] running on GCE node
W20260806 12:56:52.044577 28780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:52.044577 28783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:52.045025 28775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:52.045279 28775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:52.046448 28775 hybrid_clock.cc:648] HybridClock initialized: now 1786021012046441 us; error 40 us; skew 500 ppm
Aug 06 12:56:52 dist-test-slave-zkpd krb5kdc[28529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021012, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:52.048913 28775 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:56:52.049325 28775 webserver.cc:460] Webserver started at http://127.24.83.2:40231/ using document root <none> and password file <none>
I20260806 12:56:52.049564 28775 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:52.049630 28775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:52.049778 28775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:52.050812 28775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "60d66acf608a465b9fe784b9c572aee4"
format_stamp: "Formatted at 2026-08-06 12:56:52 on dist-test-slave-zkpd"
server_key: "1785b5245ac0b7dadc60e6356265f646"
server_key_iv: "bb97f51ff5e41d43064a15b1c46554ab"
server_key_version: "encryptionkey@0"
I20260806 12:56:52.051172 28775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "60d66acf608a465b9fe784b9c572aee4"
format_stamp: "Formatted at 2026-08-06 12:56:52 on dist-test-slave-zkpd"
server_key: "1785b5245ac0b7dadc60e6356265f646"
server_key_iv: "bb97f51ff5e41d43064a15b1c46554ab"
server_key_version: "encryptionkey@0"
I20260806 12:56:52.052531 28775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 12:56:52.053305 28790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:52.053493 28775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 12:56:52.053598 28775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "60d66acf608a465b9fe784b9c572aee4"
format_stamp: "Formatted at 2026-08-06 12:56:52 on dist-test-slave-zkpd"
server_key: "1785b5245ac0b7dadc60e6356265f646"
server_key_iv: "bb97f51ff5e41d43064a15b1c46554ab"
server_key_version: "encryptionkey@0"
I20260806 12:56:52.053673 28775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:52.069072 28775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:52.070776 28775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:52.070945 28775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:52.071164 28775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:52.071460 28775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:52.071512 28775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:52.071544 28775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:52.071571 28775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:52.077358 28775 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:37631
I20260806 12:56:52.077432 28905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:37631 every 8 connection(s)
I20260806 12:56:52.077739 28775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:56:52.081089 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 28775
I20260806 12:56:52.081202 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:56:52.081386 24908 external_mini_cluster.cc:1428] Setting key 3daf9f0e70ea9df0f64acc1f484fdc6c
Aug 06 12:56:52 dist-test-slave-zkpd krb5kdc[28529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021012, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:52.088132 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:52.078397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40851 (local address 127.24.83.2:35299)
0806 12:56:52.078561 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:52.078601 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:52.079482 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:52.079691 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:52.079693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:52.080009 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:52.080190 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:52.080195 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.080853 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.080854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:52.082102 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:52.082105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.082456 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.082457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:52.082599 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:52.083127 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:52.083137 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:52.084928 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:52.086778 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:52.086785 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:52.086786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:52.087000 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:52.087223 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:52.087223 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:52.087223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:52.087315 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:52.087606 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:52.087610 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:52.087716 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 12:56:52.087794 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:56:52.087976 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260806 12:56:52.088127 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:52.078444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:35299 (local address 127.24.83.62:40851)
0806 12:56:52.078495 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:52.078497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:52.078699 (+ 202us) server_negotiation.cc:407] Connection header received
0806 12:56:52.079572 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:52.079573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:52.079582 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:52.079638 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:52.080272 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.080770 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.082188 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.082374 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.082426 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:52.085025 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:52.085026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:52.085027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:52.085035 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:52.086675 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:52.087080 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:52.087082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:52.087083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:52.087148 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:52.087384 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:52.087385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:52.087385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:52.087537 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:52.087614 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:52.087846 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 12:56:52.087982 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 12:56:52.088665 28906 heartbeater.cc:347] Connected to a master server at 127.24.83.62:40851
I20260806 12:56:52.088771 28906 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:52.088963 28906 heartbeater.cc:511] Master 127.24.83.62:40851 requested a full tablet report, sending...
I20260806 12:56:52.089388 28575 ts_manager.cc:194] Registered new tserver with Master: 60d66acf608a465b9fe784b9c572aee4 (127.24.83.2:37631)
I20260806 12:56:52.089915 28575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:35299
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:56:52.106974 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40851
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:33533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:52.188369 28911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:52.188550 28911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:52.188573 28911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:52.189708 28911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:52.189749 28911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:52.189774 28911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:56:52.191267 28911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:33533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:52.191448 28911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:52.191615 28911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:52.196305 28918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:52.196427 28911 server_base.cc:1034] running on GCE node
W20260806 12:56:52.196336 28920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:52.196368 28917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:52.196750 28911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:52.197002 28911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:52.198170 28911 hybrid_clock.cc:648] HybridClock initialized: now 1786021012198159 us; error 40 us; skew 500 ppm
Aug 06 12:56:52 dist-test-slave-zkpd krb5kdc[28529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021012, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:52.200649 28911 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:56:52.201083 28911 webserver.cc:460] Webserver started at http://127.24.83.3:33013/ using document root <none> and password file <none>
I20260806 12:56:52.201321 28911 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:52.201397 28911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:52.201542 28911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:52.202787 28911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "e78a6e35f0bc453ea24b94802773c998"
format_stamp: "Formatted at 2026-08-06 12:56:52 on dist-test-slave-zkpd"
server_key: "bd6df820414eb934b8bfc7e9a76ded28"
server_key_iv: "2b51597551052d3921ecbd1e3e8d5e93"
server_key_version: "encryptionkey@0"
I20260806 12:56:52.203197 28911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "e78a6e35f0bc453ea24b94802773c998"
format_stamp: "Formatted at 2026-08-06 12:56:52 on dist-test-slave-zkpd"
server_key: "bd6df820414eb934b8bfc7e9a76ded28"
server_key_iv: "2b51597551052d3921ecbd1e3e8d5e93"
server_key_version: "encryptionkey@0"
I20260806 12:56:52.204624 28911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:52.205363 28927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:52.205530 28911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:52.205636 28911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "e78a6e35f0bc453ea24b94802773c998"
format_stamp: "Formatted at 2026-08-06 12:56:52 on dist-test-slave-zkpd"
server_key: "bd6df820414eb934b8bfc7e9a76ded28"
server_key_iv: "2b51597551052d3921ecbd1e3e8d5e93"
server_key_version: "encryptionkey@0"
I20260806 12:56:52.205716 28911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:52.237823 28911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:52.239670 28911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:52.239876 28911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:52.240195 28911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:52.240522 28911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:52.240563 28911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:52.240585 28911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:52.240636 28911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:52.246377 28911 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:35109
I20260806 12:56:52.246438 29047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:35109 every 8 connection(s)
I20260806 12:56:52.246742 28911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:56:52.251693 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 28911
I20260806 12:56:52.251777 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:56:52.252031 24908 external_mini_cluster.cc:1428] Setting key 9747d20a6b64931e9295edc38d47c702
Aug 06 12:56:52 dist-test-slave-zkpd krb5kdc[28529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021012, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:52.255388 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:52.247503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:38127 (local address 127.24.83.62:40851)
0806 12:56:52.247571 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:52.247572 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:52.247760 (+ 188us) server_negotiation.cc:407] Connection header received
0806 12:56:52.248670 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:52.248671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:52.248682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:52.248735 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:52.249308 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.249789 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.250731 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.250908 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.250957 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:52.253111 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:52.253112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:52.253112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:52.253118 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:52.254306 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:52.254657 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:52.254657 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:52.254658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:52.254706 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:52.254918 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:52.254919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:52.254919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:52.255011 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:52.255060 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:52.255264 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 12:56:52.255328 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 12:56:52.255645 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:52.247474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40851 (local address 127.24.83.3:38127)
0806 12:56:52.247626 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:52.247667 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:52.248582 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:52.248802 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:52.248805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:52.249049 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:52.249234 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:52.249238 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.249871 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.249871 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:52.250649 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:52.250651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.250980 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.250981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:52.251076 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:52.251480 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:52.251487 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:52.253033 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:52.254374 (+ 1341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:52.254380 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:52.254382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:52.254591 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:52.254764 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:52.254764 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:52.254765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:52.254864 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:52.255066 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:52.255070 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:52.255174 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 12:56:52.255252 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:56:52.255380 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260806 12:56:52.256126 29049 heartbeater.cc:347] Connected to a master server at 127.24.83.62:40851
I20260806 12:56:52.256237 29049 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:52.256407 29049 heartbeater.cc:511] Master 127.24.83.62:40851 requested a full tablet report, sending...
I20260806 12:56:52.256768 28575 ts_manager.cc:194] Registered new tserver with Master: e78a6e35f0bc453ea24b94802773c998 (127.24.83.3:35109)
I20260806 12:56:52.257133 28575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:38127
I20260806 12:56:52.263768 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 12:56:52 dist-test-slave-zkpd krb5kdc[28529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 12:56:52 dist-test-slave-zkpd krb5kdc[28529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:56:52.286554 29056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:52.278694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.24.83.1:43483)
0806 12:56:52.278901 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:52.278907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:52.278918 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:56:52.278951 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:52.278953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:52.278967 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:52.279054 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:52.279615 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.280429 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.281201 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.281537 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.281619 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:52.283823 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:52.283832 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:52.283833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:52.283840 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:52.285326 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:52.285687 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:52.285690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:52.285690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:52.285758 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:52.285960 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:52.285961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:52.285961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:52.286108 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:52.286267 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:52.286349 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 12:56:52.286431 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
Aug 06 12:56:52 dist-test-slave-zkpd krb5kdc[28529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:52.294226 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:52.287808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.24.83.62:40851)
0806 12:56:52.287862 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:52.287864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:52.287874 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:56:52.287968 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:52.287969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:52.287980 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:52.288029 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:52.288474 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.288970 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.289645 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:52.289869 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:52.289947 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:52.292070 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:52.292072 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:52.292072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:52.292079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:52.293244 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:52.293563 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:52.293564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:52.293565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:52.293614 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:52.293787 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:52.293787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:52.293788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:52.293936 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:52.294009 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:52.294098 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 12:56:52.294159 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 12:56:52.294497 28575 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51804
I20260806 12:56:52.294900 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 28638
I20260806 12:56:52.300252 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 28775
I20260806 12:56:52.305434 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 28911
I20260806 12:56:52.311928 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 28542
2026-08-06T12:56:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1841 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:56:52 dist-test-slave-zkpd krb5kdc[29061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:56:52 dist-test-slave-zkpd krb5kdc[29061](info): set up 2 sockets
Aug 06 12:56:52 dist-test-slave-zkpd krb5kdc[29061](info): commencing operation
krb5kdc: starting...
W20260806 12:56:53.360198 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:56:53 dist-test-slave-zkpd krb5kdc[29061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:56:53.476228 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:43323
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:43323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:53.555975 29075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:53.556166 29075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:53.556187 29075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:53.557349 29075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:53.557394 29075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:53.557408 29075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:53.557421 29075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:53.557482 29075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:53.558948 29075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:43323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:43323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:53.559198 29075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:53.559430 29075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:53.564103 29081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:53.564141 29075 server_base.cc:1034] running on GCE node
W20260806 12:56:53.564150 29082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:53.564123 29084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:53.564553 29075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:53.564857 29075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:53.566022 29075 hybrid_clock.cc:648] HybridClock initialized: now 1786021013566008 us; error 45 us; skew 500 ppm
Aug 06 12:56:53 dist-test-slave-zkpd krb5kdc[29061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021013, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:53.568459 29075 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:56:53.568822 29075 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:53.568873 29075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:53.568961 29075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:53.569936 29075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "5d8152716f40440aa4b4eb17b3ea9154"
format_stamp: "Formatted at 2026-08-06 12:56:53 on dist-test-slave-zkpd"
server_key: "f4a6c4316bcbc272ed05569deb61c65a"
server_key_iv: "1171dd1bca38cc3d91ee3b695d9ca76c"
server_key_version: "encryptionkey@0"
I20260806 12:56:53.570292 29075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "5d8152716f40440aa4b4eb17b3ea9154"
format_stamp: "Formatted at 2026-08-06 12:56:53 on dist-test-slave-zkpd"
server_key: "f4a6c4316bcbc272ed05569deb61c65a"
server_key_iv: "1171dd1bca38cc3d91ee3b695d9ca76c"
server_key_version: "encryptionkey@0"
I20260806 12:56:53.571568 29075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:56:53.572343 29090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:53.572520 29075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:53.572625 29075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "5d8152716f40440aa4b4eb17b3ea9154"
format_stamp: "Formatted at 2026-08-06 12:56:53 on dist-test-slave-zkpd"
server_key: "f4a6c4316bcbc272ed05569deb61c65a"
server_key_iv: "1171dd1bca38cc3d91ee3b695d9ca76c"
server_key_version: "encryptionkey@0"
I20260806 12:56:53.572700 29075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:53.613588 29075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:53.615502 29075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:53.615676 29075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:53.619441 29075 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:43323
I20260806 12:56:53.619513 29146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:43323 every 8 connection(s)
I20260806 12:56:53.619807 29075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:53.620983 29147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:53.621026 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 29075
I20260806 12:56:53.621132 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:53.621341 24908 external_mini_cluster.cc:1428] Setting key de8cee1b41e1e858c72f7cb7c14bec70
I20260806 12:56:53.624025 29147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154: Bootstrap starting.
I20260806 12:56:53.624625 29147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:53.624920 29147 log.cc:824] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:53.625576 29147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154: No bootstrap required, opened a new log
I20260806 12:56:53.627388 29147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8152716f40440aa4b4eb17b3ea9154" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43323 } }
I20260806 12:56:53.627559 29147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:53.627599 29147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d8152716f40440aa4b4eb17b3ea9154, State: Initialized, Role: FOLLOWER
I20260806 12:56:53.627730 29147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8152716f40440aa4b4eb17b3ea9154" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43323 } }
I20260806 12:56:53.627806 29147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 06 12:56:53 dist-test-slave-zkpd krb5kdc[29061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:53.627846 29147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:53.627938 29147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:53.628484 29147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8152716f40440aa4b4eb17b3ea9154" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43323 } }
I20260806 12:56:53.628597 29147 leader_election.cc:302] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d8152716f40440aa4b4eb17b3ea9154; no voters:
I20260806 12:56:53.628762 29147 leader_election.cc:288] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:53.628917 29152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:53.629019 29147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:53.629114 29152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [term 1 LEADER]: Becoming Leader. State: Replica: 5d8152716f40440aa4b4eb17b3ea9154, State: Running, Role: LEADER
I20260806 12:56:53.629541 29152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8152716f40440aa4b4eb17b3ea9154" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43323 } }
I20260806 12:56:53.630240 29153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d8152716f40440aa4b4eb17b3ea9154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8152716f40440aa4b4eb17b3ea9154" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43323 } } }
I20260806 12:56:53.630327 29153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:53.630434 29152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d8152716f40440aa4b4eb17b3ea9154. Latest consensus state: current_term: 1 leader_uuid: "5d8152716f40440aa4b4eb17b3ea9154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8152716f40440aa4b4eb17b3ea9154" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43323 } } }
I20260806 12:56:53.630676 29152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154 [sys.catalog]: This master's current role is: LEADER
W20260806 12:56:53.630937 29167 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:56:53.631018 29167 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:56:53.631174 29168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:53.631390 29168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:53.632680 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:53.621913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50872 (local address 127.24.83.62:43323)
0806 12:56:53.622093 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:53.622109 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:53.622134 (+ 25us) server_negotiation.cc:407] Connection header received
0806 12:56:53.622570 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:53.622577 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:53.622621 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:53.622771 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:53.623407 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:53.624426 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:53.625326 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:53.625770 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:53.625903 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:53.628371 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:53.628376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:53.628376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:53.628388 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:53.630673 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:53.631088 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:53.631092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:53.631093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:53.631199 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:53.631462 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:53.631462 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:53.631462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:53.631745 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:53.632279 (+ 534us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:53.632355 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 12:56:53.632513 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260806 12:56:53.633026 29168 catalog_manager.cc:1347] Generated new cluster ID: 982e83c726f648348cb335dccee83d8a
I20260806 12:56:53.633064 29168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:56:53.653980 29168 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:53.654714 29168 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:53.671694 29168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154: Generated new TSK 0
I20260806 12:56:53.671908 29168 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:56:53.704738 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43323
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:53.785723 29174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:53.785952 29174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:53.786005 29174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:53.787154 29174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:53.787220 29174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:53.787267 29174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:56:53.788817 29174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:53.789091 29174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:53.789310 29174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:53.794157 29184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:53.794214 29180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:53.794392 29174 server_base.cc:1034] running on GCE node
W20260806 12:56:53.794157 29182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:53.794658 29174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:53.794915 29174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:53.796075 29174 hybrid_clock.cc:648] HybridClock initialized: now 1786021013796063 us; error 44 us; skew 500 ppm
Aug 06 12:56:53 dist-test-slave-zkpd krb5kdc[29061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021013, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:53.798564 29174 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:56:53.798945 29174 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:53.799015 29174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:53.799151 29174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:53.800259 29174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "8cc5942fbb6d48eea13347d37a3c215d"
format_stamp: "Formatted at 2026-08-06 12:56:53 on dist-test-slave-zkpd"
server_key: "ba7c640306cd08774da1c44cf5d19c77"
server_key_iv: "19be4f56cc2d531080c195239f03f36e"
server_key_version: "encryptionkey@0"
I20260806 12:56:53.800616 29174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "8cc5942fbb6d48eea13347d37a3c215d"
format_stamp: "Formatted at 2026-08-06 12:56:53 on dist-test-slave-zkpd"
server_key: "ba7c640306cd08774da1c44cf5d19c77"
server_key_iv: "19be4f56cc2d531080c195239f03f36e"
server_key_version: "encryptionkey@0"
I20260806 12:56:53.801877 29174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:56:53.802565 29190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:53.802735 29174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:53.802842 29174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "8cc5942fbb6d48eea13347d37a3c215d"
format_stamp: "Formatted at 2026-08-06 12:56:53 on dist-test-slave-zkpd"
server_key: "ba7c640306cd08774da1c44cf5d19c77"
server_key_iv: "19be4f56cc2d531080c195239f03f36e"
server_key_version: "encryptionkey@0"
I20260806 12:56:53.802922 29174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:53.833381 29174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:53.835883 29174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:53.836084 29174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:53.836347 29174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:53.836691 29174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:53.836745 29174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:53.836795 29174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:53.836843 29174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:53.842751 29174 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:40179
I20260806 12:56:53.842800 29308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:40179 every 8 connection(s)
I20260806 12:56:53.843070 29174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
Aug 06 12:56:53 dist-test-slave-zkpd krb5kdc[29061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021013, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:53.849613 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 29174
I20260806 12:56:53.849736 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:56:53.849967 24908 external_mini_cluster.cc:1428] Setting key 90564e292ce7225d678bee66dffbb65d
I20260806 12:56:53.852660 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:53.843863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:57039 (local address 127.24.83.62:43323)
0806 12:56:53.843955 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:53.843957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:53.844143 (+ 186us) server_negotiation.cc:407] Connection header received
0806 12:56:53.844676 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:53.844677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:53.844691 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:53.844743 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:53.845696 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:53.846214 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:53.847215 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:53.847450 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:53.847543 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:53.849680 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:53.849682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:53.849682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:53.849690 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:53.851228 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:53.851635 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:53.851636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:53.851637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:53.851699 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:53.851988 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:53.851989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:53.851990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:53.852111 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:53.852190 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:53.852473 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 12:56:53.852572 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 12:56:53.852823 29311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:53.843847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:43323 (local address 127.24.83.1:57039)
0806 12:56:53.844024 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:53.844061 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:53.844559 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:53.844825 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:53.844829 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:53.845080 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:53.845581 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:53.845586 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:53.846320 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:53.846321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:53.847118 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:53.847120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:53.847553 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:53.847554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:53.847673 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:53.848222 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:53.848231 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:53.849578 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:53.851319 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:53.851326 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:53.851328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:53.851548 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:53.851776 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:53.851776 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:53.851777 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:53.851931 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:53.852182 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:53.852188 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:53.852336 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 12:56:53.852436 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 12:56:53.852591 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260806 12:56:53.853317 29309 heartbeater.cc:347] Connected to a master server at 127.24.83.62:43323
I20260806 12:56:53.853413 29309 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:53.853574 29309 heartbeater.cc:511] Master 127.24.83.62:43323 requested a full tablet report, sending...
I20260806 12:56:53.854089 29109 ts_manager.cc:194] Registered new tserver with Master: 8cc5942fbb6d48eea13347d37a3c215d (127.24.83.1:40179)
I20260806 12:56:53.854712 29109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:57039
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:56:53.874555 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43323
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:53.954824 29314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:53.955021 29314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:53.955044 29314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:53.956146 29314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:53.956179 29314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:53.956204 29314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:56:53.957630 29314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:53.957815 29314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:53.957983 29314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:53.962829 29320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:53.962841 29319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:53.962842 29322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:53.963006 29314 server_base.cc:1034] running on GCE node
I20260806 12:56:53.963306 29314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:53.963548 29314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:53.964685 29314 hybrid_clock.cc:648] HybridClock initialized: now 1786021013964674 us; error 29 us; skew 500 ppm
Aug 06 12:56:53 dist-test-slave-zkpd krb5kdc[29061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021013, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:53.967029 29314 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:56:53.967293 29314 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:53.967334 29314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:53.967425 29314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:53.968421 29314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "9374809172d546e7b44146a9b61235e1"
format_stamp: "Formatted at 2026-08-06 12:56:53 on dist-test-slave-zkpd"
server_key: "48eab805a3ee331dcfe9fce366b1d7fa"
server_key_iv: "1e573427d43b1e5d28e2cdbbc1b8a053"
server_key_version: "encryptionkey@0"
I20260806 12:56:53.968770 29314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "9374809172d546e7b44146a9b61235e1"
format_stamp: "Formatted at 2026-08-06 12:56:53 on dist-test-slave-zkpd"
server_key: "48eab805a3ee331dcfe9fce366b1d7fa"
server_key_iv: "1e573427d43b1e5d28e2cdbbc1b8a053"
server_key_version: "encryptionkey@0"
I20260806 12:56:53.969967 29314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:56:53.970620 29328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:53.970809 29314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:53.970891 29314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "9374809172d546e7b44146a9b61235e1"
format_stamp: "Formatted at 2026-08-06 12:56:53 on dist-test-slave-zkpd"
server_key: "48eab805a3ee331dcfe9fce366b1d7fa"
server_key_iv: "1e573427d43b1e5d28e2cdbbc1b8a053"
server_key_version: "encryptionkey@0"
I20260806 12:56:53.970943 29314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:53.984541 29314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:53.986274 29314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:53.986392 29314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:53.986574 29314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:53.986821 29314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:53.986852 29314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:53.986874 29314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:53.986887 29314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:53.992660 29314 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:34449
I20260806 12:56:53.992727 29446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:34449 every 8 connection(s)
I20260806 12:56:53.992986 29314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:56:53 dist-test-slave-zkpd krb5kdc[29061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021013, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:53.999157 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 29314
I20260806 12:56:53.999274 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:56:53.999498 24908 external_mini_cluster.cc:1428] Setting key 62c0922f89c41937e5c3d6c94c9bfdd0
I20260806 12:56:54.002234 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:53.993694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:33625 (local address 127.24.83.62:43323)
0806 12:56:53.993765 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:53.993766 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:53.993983 (+ 217us) server_negotiation.cc:407] Connection header received
0806 12:56:53.994439 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:53.994440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:53.994449 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:53.994506 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:53.995349 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:53.995834 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:53.996824 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:53.997007 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:53.997060 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:53.999146 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:53.999147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:53.999148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:53.999156 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:54.000772 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:54.001260 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:54.001261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:54.001261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:54.001311 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:54.001571 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:54.001572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:54.001572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:54.001699 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:54.001776 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.002058 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.002143 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:56:54.002431 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:53.993662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:43323 (local address 127.24.83.2:33625)
0806 12:56:53.993846 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:53.993893 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:53.994355 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:53.994556 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:53.994559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:53.994800 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:53.995276 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:53.995281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:53.995960 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:53.995961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:53.996745 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:53.996747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:53.997080 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:53.997081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:53.997176 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:53.997693 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:53.997701 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:53.999050 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:54.000861 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:54.000868 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:54.000870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:54.001186 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:54.001372 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:54.001373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:54.001374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:54.001511 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:54.001770 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:54.001778 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:54.001921 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 12:56:54.002032 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 12:56:54.002183 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260806 12:56:54.002925 29447 heartbeater.cc:347] Connected to a master server at 127.24.83.62:43323
I20260806 12:56:54.003031 29447 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:54.003193 29447 heartbeater.cc:511] Master 127.24.83.62:43323 requested a full tablet report, sending...
I20260806 12:56:54.003564 29109 ts_manager.cc:194] Registered new tserver with Master: 9374809172d546e7b44146a9b61235e1 (127.24.83.2:34449)
I20260806 12:56:54.004004 29109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:33625
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:56:54.026757 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43323
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:54.109715 29453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:54.109920 29453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:54.109943 29453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:54.111061 29453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:54.111099 29453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:54.111124 29453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:56:54.112660 29453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:54.112903 29453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:54.113126 29453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:56:54.118073 29453 server_base.cc:1034] running on GCE node
W20260806 12:56:54.118026 29459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:54.118054 29461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:54.118021 29458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:54.118429 29453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:54.118671 29453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:54.119815 29453 hybrid_clock.cc:648] HybridClock initialized: now 1786021014119807 us; error 27 us; skew 500 ppm
Aug 06 12:56:54 dist-test-slave-zkpd krb5kdc[29061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021014, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:54.122354 29453 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:56:54.122705 29453 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:54.122754 29453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:54.122851 29453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:54.123845 29453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "6fd87a60f6124226a0d02a095cabb640"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "fd06c97e71ccb67602c681a44c1cc5af"
server_key_iv: "6b5a2fe1627253651bfdc0cc97d13afb"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.124218 29453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "6fd87a60f6124226a0d02a095cabb640"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "fd06c97e71ccb67602c681a44c1cc5af"
server_key_iv: "6b5a2fe1627253651bfdc0cc97d13afb"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.125486 29453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:56:54.126190 29467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:54.126361 29453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:54.126448 29453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "6fd87a60f6124226a0d02a095cabb640"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "fd06c97e71ccb67602c681a44c1cc5af"
server_key_iv: "6b5a2fe1627253651bfdc0cc97d13afb"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.126515 29453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:54.140177 29453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:54.141992 29453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:54.142145 29453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:54.142361 29453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:54.142673 29453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:54.142726 29453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:54.142766 29453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:54.142793 29453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:54.148406 29453 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:38811
I20260806 12:56:54.148442 29583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:38811 every 8 connection(s)
I20260806 12:56:54.148738 29453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:56:54.151770 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 29453
I20260806 12:56:54.151877 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:56:54.152149 24908 external_mini_cluster.cc:1428] Setting key d72ce3545be69c5c28ecab8e6636ef85
Aug 06 12:56:54 dist-test-slave-zkpd krb5kdc[29061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021014, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:54.158017 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.149490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:34463 (local address 127.24.83.62:43323)
0806 12:56:54.149556 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:54.149558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:54.149762 (+ 204us) server_negotiation.cc:407] Connection header received
0806 12:56:54.150218 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:54.150220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:54.150229 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:54.150284 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:54.151169 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.151671 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.153173 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.153359 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.153413 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.155431 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:54.155433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:54.155433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:54.155439 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:54.156914 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:54.157256 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:54.157257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:54.157257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:54.157307 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:54.157509 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:54.157510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:54.157510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:54.157606 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:54.157655 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.157884 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.157954 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:56:54.158161 29587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.149479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:43323 (local address 127.24.83.3:34463)
0806 12:56:54.149625 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:54.149667 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:54.150138 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:54.150348 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:54.150351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:54.150598 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:54.151099 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.151103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.151761 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.151762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.153102 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.153105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.153433 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.153433 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.153535 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.153983 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:54.153991 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:54.155344 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:54.156984 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:54.156989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:54.156991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:54.157193 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:54.157364 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:54.157365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:54.157365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:54.157456 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:54.157659 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:54.157663 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:54.157768 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 12:56:54.157857 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 12:56:54.157970 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260806 12:56:54.158612 29584 heartbeater.cc:347] Connected to a master server at 127.24.83.62:43323
I20260806 12:56:54.158723 29584 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:54.158913 29584 heartbeater.cc:511] Master 127.24.83.62:43323 requested a full tablet report, sending...
I20260806 12:56:54.159291 29109 ts_manager.cc:194] Registered new tserver with Master: 6fd87a60f6124226a0d02a095cabb640 (127.24.83.3:38811)
I20260806 12:56:54.159648 29109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:34463
I20260806 12:56:54.163568 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:56:54.170697 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.164906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.24.83.62:43323)
0806 12:56:54.164985 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:54.164987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:54.165077 (+ 90us) server_negotiation.cc:407] Connection header received
0806 12:56:54.165145 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:54.165146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:54.165156 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:54.165210 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:54.165787 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.166256 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.166972 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.167173 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.167248 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.168395 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:54.168396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:54.168397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:54.168404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:54.169528 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:54.169883 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:54.169884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:54.169885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:54.169943 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:54.170147 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:54.170147 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:54.170148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:54.170292 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:54.170399 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.170520 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.170596 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 12:56:54.172429 29109 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:56:54.174103 29109 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:56:54.180285 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.176369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:40179 (local address 127.0.0.1:39366)
0806 12:56:54.176529 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:54.176574 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:54.176639 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:54.176835 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:54.176838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:54.176839 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:54.176917 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.176919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.178265 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.178266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.179180 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.179182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.180004 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.180005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.180078 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.180092 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 12:56:54.180163 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 12:56:54.180210 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":32,"thread_start_us":47,"threads_started":1}
I20260806 12:56:54.180306 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.176482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39366 (local address 127.24.83.1:40179)
0806 12:56:54.176674 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:54.176679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:54.176689 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:56:54.176730 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:54.176732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:54.176747 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:54.176829 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:54.177014 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.178156 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.179291 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.179926 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.180003 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.180057 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.180181 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.180218 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":79,"threads_started":1}
I20260806 12:56:54.180675 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.176369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:38811 (local address 127.0.0.1:40436)
0806 12:56:54.176673 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:54.176685 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:54.176737 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:54.177324 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:54.177324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:54.177327 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:54.177376 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.177377 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.178234 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.178235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.179517 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.179519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.180447 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.180448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.180516 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.180521 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:56:54.180565 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 12:56:54.180607 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":86,"thread_start_us":86,"threads_started":1}
I20260806 12:56:54.180759 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.176470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40436 (local address 127.24.83.3:38811)
0806 12:56:54.177134 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:54.177138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:54.177148 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:56:54.177191 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:54.177193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:54.177208 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:54.177451 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:54.177507 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.178135 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.179600 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.180341 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.180440 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.180596 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.180621 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.180661 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":76,"threads_started":1}
I20260806 12:56:54.181137 29239 tablet_service.cc:1467] Processing CreateTablet for tablet fd0adf19b97e488e8b690669c83552da (DEFAULT_TABLE table=test-table [id=815f681c04694b219d4cb6acd9ae6a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:54.181456 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.176559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:34449 (local address 127.0.0.1:43554)
0806 12:56:54.176944 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:54.176954 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:54.176995 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:54.177252 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:54.177253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:54.177255 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:54.177309 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.177310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.178499 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.178500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.179752 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.179754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.181214 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.181215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.181278 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.181283 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:56:54.181349 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 12:56:54.181388 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":77,"threads_started":1}
I20260806 12:56:54.181497 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.176798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43554 (local address 127.24.83.2:34449)
0806 12:56:54.176979 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:54.176983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:54.176995 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:56:54.177066 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:54.177068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:54.177085 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:54.177161 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:54.177390 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.178396 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.179839 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.181122 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.181204 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.181258 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.181365 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.181407 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260806 12:56:54.181531 29239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0adf19b97e488e8b690669c83552da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:54.181502 29515 tablet_service.cc:1467] Processing CreateTablet for tablet fd0adf19b97e488e8b690669c83552da (DEFAULT_TABLE table=test-table [id=815f681c04694b219d4cb6acd9ae6a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:54.181799 29515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0adf19b97e488e8b690669c83552da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:54.182065 29377 tablet_service.cc:1467] Processing CreateTablet for tablet fd0adf19b97e488e8b690669c83552da (DEFAULT_TABLE table=test-table [id=815f681c04694b219d4cb6acd9ae6a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:54.182379 29377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0adf19b97e488e8b690669c83552da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:54.185328 29603 tablet_bootstrap.cc:492] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d: Bootstrap starting.
I20260806 12:56:54.185667 29604 tablet_bootstrap.cc:492] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: Bootstrap starting.
I20260806 12:56:54.185883 29602 tablet_bootstrap.cc:492] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: Bootstrap starting.
I20260806 12:56:54.185925 29603 tablet_bootstrap.cc:654] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:54.186393 29604 tablet_bootstrap.cc:654] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:54.186515 29603 log.cc:824] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:54.186578 29602 tablet_bootstrap.cc:654] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:54.187227 29602 log.cc:824] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:54.187247 29604 log.cc:824] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:54.187371 29603 tablet_bootstrap.cc:492] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d: No bootstrap required, opened a new log
I20260806 12:56:54.187439 29603 ts_tablet_manager.cc:1397] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:56:54.188021 29602 tablet_bootstrap.cc:492] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: No bootstrap required, opened a new log
I20260806 12:56:54.188103 29602 ts_tablet_manager.cc:1397] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:56:54.188237 29604 tablet_bootstrap.cc:492] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: No bootstrap required, opened a new log
I20260806 12:56:54.188295 29604 ts_tablet_manager.cc:1397] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 12:56:54.189385 29602 raft_consensus.cc:348] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc5942fbb6d48eea13347d37a3c215d" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40179 } } peers { permanent_uuid: "9374809172d546e7b44146a9b61235e1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 34449 } } peers { permanent_uuid: "6fd87a60f6124226a0d02a095cabb640" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 38811 } }
I20260806 12:56:54.189499 29602 raft_consensus.cc:374] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:54.189520 29602 raft_consensus.cc:729] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fd87a60f6124226a0d02a095cabb640, State: Initialized, Role: FOLLOWER
I20260806 12:56:54.189415 29603 raft_consensus.cc:348] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc5942fbb6d48eea13347d37a3c215d" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40179 } } peers { permanent_uuid: "9374809172d546e7b44146a9b61235e1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 34449 } } peers { permanent_uuid: "6fd87a60f6124226a0d02a095cabb640" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 38811 } }
I20260806 12:56:54.189565 29603 raft_consensus.cc:374] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:54.189599 29603 raft_consensus.cc:729] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc5942fbb6d48eea13347d37a3c215d, State: Initialized, Role: FOLLOWER
I20260806 12:56:54.189638 29602 consensus_queue.cc:260] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc5942fbb6d48eea13347d37a3c215d" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40179 } } peers { permanent_uuid: "9374809172d546e7b44146a9b61235e1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 34449 } } peers { permanent_uuid: "6fd87a60f6124226a0d02a095cabb640" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 38811 } }
I20260806 12:56:54.189729 29603 consensus_queue.cc:260] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc5942fbb6d48eea13347d37a3c215d" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40179 } } peers { permanent_uuid: "9374809172d546e7b44146a9b61235e1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 34449 } } peers { permanent_uuid: "6fd87a60f6124226a0d02a095cabb640" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 38811 } }
I20260806 12:56:54.189918 29584 heartbeater.cc:503] Master 127.24.83.62:43323 was elected leader, sending a full tablet report...
I20260806 12:56:54.189985 29603 ts_tablet_manager.cc:1428] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 12:56:54.189916 29602 ts_tablet_manager.cc:1428] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:56:54.190184 29309 heartbeater.cc:503] Master 127.24.83.62:43323 was elected leader, sending a full tablet report...
I20260806 12:56:54.190315 29604 raft_consensus.cc:348] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc5942fbb6d48eea13347d37a3c215d" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40179 } } peers { permanent_uuid: "9374809172d546e7b44146a9b61235e1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 34449 } } peers { permanent_uuid: "6fd87a60f6124226a0d02a095cabb640" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 38811 } }
I20260806 12:56:54.190493 29604 raft_consensus.cc:374] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:54.190577 29604 raft_consensus.cc:729] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9374809172d546e7b44146a9b61235e1, State: Initialized, Role: FOLLOWER
I20260806 12:56:54.190722 29604 consensus_queue.cc:260] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc5942fbb6d48eea13347d37a3c215d" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40179 } } peers { permanent_uuid: "9374809172d546e7b44146a9b61235e1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 34449 } } peers { permanent_uuid: "6fd87a60f6124226a0d02a095cabb640" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 38811 } }
I20260806 12:56:54.191116 29604 ts_tablet_manager.cc:1428] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 12:56:54.191314 29447 heartbeater.cc:503] Master 127.24.83.62:43323 was elected leader, sending a full tablet report...
W20260806 12:56:54.243580 29448 tablet.cc:2369] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:56:54.343824 29310 tablet.cc:2369] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:56:54.399307 29586 tablet.cc:2369] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:56:54.502743 29609 raft_consensus.cc:482] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:56:54.502893 29609 raft_consensus.cc:504] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc5942fbb6d48eea13347d37a3c215d" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40179 } } peers { permanent_uuid: "9374809172d546e7b44146a9b61235e1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 34449 } } peers { permanent_uuid: "6fd87a60f6124226a0d02a095cabb640" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 38811 } }
I20260806 12:56:54.503281 29609 leader_election.cc:288] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9374809172d546e7b44146a9b61235e1 (127.24.83.2:34449), 6fd87a60f6124226a0d02a095cabb640 (127.24.83.3:38811)
I20260806 12:56:54.506253 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.503513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:58033 (local address 127.24.83.2:34449)
0806 12:56:54.503561 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:54.503563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:54.503695 (+ 132us) server_negotiation.cc:407] Connection header received
0806 12:56:54.503783 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:54.503784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:54.503795 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:54.503851 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:54.504051 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.504523 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.505533 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.505899 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.505991 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.506002 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.506149 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.506184 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:56:54.506273 29612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.503459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:34449 (local address 127.24.83.1:58033)
0806 12:56:54.503612 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:54.503623 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:54.503714 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:54.503863 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:54.503864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:54.503866 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:54.503967 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.503971 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.504609 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.504611 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.505439 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.505442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.505995 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.505996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.506065 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.506075 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 12:56:54.506143 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 12:56:54.506183 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":1,"thread_start_us":60,"threads_started":1}
I20260806 12:56:54.506345 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.503563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:39337 (local address 127.24.83.3:38811)
0806 12:56:54.503612 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:54.503614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:54.503731 (+ 117us) server_negotiation.cc:407] Connection header received
0806 12:56:54.503808 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:54.503809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:54.503821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:54.503892 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:56:54.504077 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.504533 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.505534 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.505979 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.506059 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.506067 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.506237 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.506270 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:56:54.506367 29611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.503486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:38811 (local address 127.24.83.1:39337)
0806 12:56:54.503648 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:54.503657 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:54.503741 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:54.503921 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:54.503922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:54.503924 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:56:54.503996 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.503998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.504618 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.504619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.505441 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:54.505444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.506061 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.506062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:54.506135 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.506138 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:56:54.506205 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 12:56:54.506247 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260806 12:56:54.506718 29535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0adf19b97e488e8b690669c83552da" candidate_uuid: "8cc5942fbb6d48eea13347d37a3c215d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fd87a60f6124226a0d02a095cabb640" is_pre_election: true
I20260806 12:56:54.506891 29535 raft_consensus.cc:2454] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cc5942fbb6d48eea13347d37a3c215d in term 0.
I20260806 12:56:54.507105 29191 leader_election.cc:302] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fd87a60f6124226a0d02a095cabb640, 8cc5942fbb6d48eea13347d37a3c215d; no voters:
I20260806 12:56:54.507090 29399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0adf19b97e488e8b690669c83552da" candidate_uuid: "8cc5942fbb6d48eea13347d37a3c215d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9374809172d546e7b44146a9b61235e1" is_pre_election: true
I20260806 12:56:54.507225 29399 raft_consensus.cc:2454] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cc5942fbb6d48eea13347d37a3c215d in term 0.
I20260806 12:56:54.507246 29609 raft_consensus.cc:2781] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:56:54.507324 29609 raft_consensus.cc:482] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:56:54.507359 29609 raft_consensus.cc:3037] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:54.508070 29609 raft_consensus.cc:504] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc5942fbb6d48eea13347d37a3c215d" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40179 } } peers { permanent_uuid: "9374809172d546e7b44146a9b61235e1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 34449 } } peers { permanent_uuid: "6fd87a60f6124226a0d02a095cabb640" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 38811 } }
I20260806 12:56:54.508221 29609 leader_election.cc:288] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [CANDIDATE]: Term 1 election: Requested vote from peers 9374809172d546e7b44146a9b61235e1 (127.24.83.2:34449), 6fd87a60f6124226a0d02a095cabb640 (127.24.83.3:38811)
I20260806 12:56:54.508411 29399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0adf19b97e488e8b690669c83552da" candidate_uuid: "8cc5942fbb6d48eea13347d37a3c215d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9374809172d546e7b44146a9b61235e1"
I20260806 12:56:54.508488 29399 raft_consensus.cc:3037] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:54.508540 29535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0adf19b97e488e8b690669c83552da" candidate_uuid: "8cc5942fbb6d48eea13347d37a3c215d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fd87a60f6124226a0d02a095cabb640"
I20260806 12:56:54.508637 29535 raft_consensus.cc:3037] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:54.509222 29399 raft_consensus.cc:2454] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cc5942fbb6d48eea13347d37a3c215d in term 1.
I20260806 12:56:54.509397 29535 raft_consensus.cc:2454] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cc5942fbb6d48eea13347d37a3c215d in term 1.
I20260806 12:56:54.509402 29194 leader_election.cc:302] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cc5942fbb6d48eea13347d37a3c215d, 9374809172d546e7b44146a9b61235e1; no voters:
I20260806 12:56:54.509480 29609 raft_consensus.cc:2781] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:54.509635 29609 raft_consensus.cc:686] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [term 1 LEADER]: Becoming Leader. State: Replica: 8cc5942fbb6d48eea13347d37a3c215d, State: Running, Role: LEADER
I20260806 12:56:54.509727 29609 consensus_queue.cc:237] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc5942fbb6d48eea13347d37a3c215d" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40179 } } peers { permanent_uuid: "9374809172d546e7b44146a9b61235e1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 34449 } } peers { permanent_uuid: "6fd87a60f6124226a0d02a095cabb640" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 38811 } }
I20260806 12:56:54.510552 29109 catalog_manager.cc:5495] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cc5942fbb6d48eea13347d37a3c215d (127.24.83.1). New cstate: current_term: 1 leader_uuid: "8cc5942fbb6d48eea13347d37a3c215d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc5942fbb6d48eea13347d37a3c215d" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9374809172d546e7b44146a9b61235e1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 34449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fd87a60f6124226a0d02a095cabb640" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 38811 } health_report { overall_health: UNKNOWN } } }
I20260806 12:56:54.535442 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.533028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39378 (local address 127.24.83.1:40179)
0806 12:56:54.533071 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:54.533073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:54.533082 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:56:54.533140 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:54.533141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:54.533160 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:54.533245 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:56:54.533399 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.533882 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.534649 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:54.534849 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:54.534943 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:54.535123 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:56:54.535221 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:56:54.535312 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.535331 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.535351 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 12:56:54.537168 29535 raft_consensus.cc:1261] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640 [term 1 FOLLOWER]: Refusing update from remote peer 8cc5942fbb6d48eea13347d37a3c215d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:54.537171 29399 raft_consensus.cc:1261] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1 [term 1 FOLLOWER]: Refusing update from remote peer 8cc5942fbb6d48eea13347d37a3c215d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:54.537544 29613 consensus_queue.cc:1035] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fd87a60f6124226a0d02a095cabb640" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:54.537683 29609 consensus_queue.cc:1035] T fd0adf19b97e488e8b690669c83552da P 8cc5942fbb6d48eea13347d37a3c215d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9374809172d546e7b44146a9b61235e1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 34449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:54.539980 29616 mvcc.cc:205] Tried to move back new op lower bound from 7315542075541770240 to 7315542075432464384. Current Snapshot: MvccSnapshot[applied={T|T < 7315542075541770240}]
I20260806 12:56:54.540622 29618 mvcc.cc:205] Tried to move back new op lower bound from 7315542075541770240 to 7315542075432464384. Current Snapshot: MvccSnapshot[applied={T|T < 7315542075541770240}]
I20260806 12:56:54.542790 29109 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:54.542886 29109 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:54.543638 29109 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d8152716f40440aa4b4eb17b3ea9154: Sending DeleteTablet for 3 replicas of tablet fd0adf19b97e488e8b690669c83552da
I20260806 12:56:54.544042 29239 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0adf19b97e488e8b690669c83552da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:54 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:39366
I20260806 12:56:54.544034 29377 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0adf19b97e488e8b690669c83552da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:54 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:43554
I20260806 12:56:54.544077 29515 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0adf19b97e488e8b690669c83552da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:54 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:40436
I20260806 12:56:54.544458 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 29174
I20260806 12:56:54.545639 29629 tablet_replica.cc:330] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: stopping tablet replica
I20260806 12:56:54.545720 29629 raft_consensus.cc:2229] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:54.545809 29629 raft_consensus.cc:2258] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:54.546010 29628 tablet_replica.cc:330] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: stopping tablet replica
I20260806 12:56:54.546097 29628 raft_consensus.cc:2229] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:54.546186 29628 raft_consensus.cc:2258] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:54.546231 29629 ts_tablet_manager.cc:1905] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:54.546726 29628 ts_tablet_manager.cc:1905] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:54.548090 29629 ts_tablet_manager.cc:1918] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:56:54.548202 29629 log.cc:1196] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/fd0adf19b97e488e8b690669c83552da
I20260806 12:56:54.548379 29628 ts_tablet_manager.cc:1918] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:56:54.548460 29628 log.cc:1196] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/fd0adf19b97e488e8b690669c83552da
I20260806 12:56:54.548537 29629 ts_tablet_manager.cc:1939] T fd0adf19b97e488e8b690669c83552da P 9374809172d546e7b44146a9b61235e1: Deleting consensus metadata
I20260806 12:56:54.548817 29628 ts_tablet_manager.cc:1939] T fd0adf19b97e488e8b690669c83552da P 6fd87a60f6124226a0d02a095cabb640: Deleting consensus metadata
I20260806 12:56:54.549014 29094 catalog_manager.cc:4841] TS 9374809172d546e7b44146a9b61235e1 (127.24.83.2:34449): tablet fd0adf19b97e488e8b690669c83552da (table test-table [id=815f681c04694b219d4cb6acd9ae6a80]) successfully deleted
I20260806 12:56:54.549515 29091 catalog_manager.cc:4841] TS 6fd87a60f6124226a0d02a095cabb640 (127.24.83.3:38811): tablet fd0adf19b97e488e8b690669c83552da (table test-table [id=815f681c04694b219d4cb6acd9ae6a80]) successfully deleted
W20260806 12:56:54.551194 29094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:40179 (error 108)
I20260806 12:56:54.551384 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 29314
I20260806 12:56:54.552222 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.552063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:40179 (local address 127.0.0.1:39384)
0806 12:56:54.552128 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:54.552180 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:40179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 12:56:54.552376 29094 catalog_manager.cc:4584] TS 8cc5942fbb6d48eea13347d37a3c215d (127.24.83.1:40179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fd0adf19b97e488e8b690669c83552da: Network error: Client connection negotiation failed: client connection to 127.24.83.1:40179: connect: Connection refused (error 111)
I20260806 12:56:54.557493 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 29453
I20260806 12:56:54.562932 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 29075
2026-08-06T12:56:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2251 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T12:56:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:54Z Disabled control of system clock
I20260806 12:56:54.590606 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:35197
--webserver_interface=127.24.83.62
--webserver_port=0
--builtin_ntp_servers=127.24.83.20:37905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:35197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 12:56:54.669286 29634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:54.669481 29634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:54.669502 29634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:54.670635 29634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:54.670674 29634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:54.670687 29634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:54.670730 29634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:54.670761 29634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:54.672070 29634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:37905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:35197
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:35197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:54.672319 29634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:54.672613 29634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:54.677277 29640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:54.677271 29642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:54.677268 29639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:54.677479 29634 server_base.cc:1034] running on GCE node
I20260806 12:56:54.677762 29634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:54.678031 29634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:54.679184 29634 hybrid_clock.cc:648] HybridClock initialized: now 1786021014679174 us; error 29 us; skew 500 ppm
I20260806 12:56:54.680397 29634 webserver.cc:460] Webserver started at http://127.24.83.62:37023/ using document root <none> and password file <none>
I20260806 12:56:54.680644 29634 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:54.680689 29634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:54.680835 29634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:54.681836 29634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "126afd6028fb483593351e35ace19513"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "062bbc747460ec92623924f107a38fb3"
server_key_iv: "6c79ab84f7b62202f06dd8340fa40e20"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.682199 29634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "126afd6028fb483593351e35ace19513"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "062bbc747460ec92623924f107a38fb3"
server_key_iv: "6c79ab84f7b62202f06dd8340fa40e20"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.683482 29634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:54.684329 29648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:54.684494 29634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 12:56:54.684583 29634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "126afd6028fb483593351e35ace19513"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "062bbc747460ec92623924f107a38fb3"
server_key_iv: "6c79ab84f7b62202f06dd8340fa40e20"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.684655 29634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:54.688254 29634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:54.688542 29634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:54.688684 29634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:54.692526 29634 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:35197
I20260806 12:56:54.692556 29702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:35197 every 8 connection(s)
I20260806 12:56:54.692866 29634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:54.693483 29703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:54.694417 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 29634
I20260806 12:56:54.694502 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:54.694681 24908 external_mini_cluster.cc:1428] Setting key 2c01965e5e4ac6b848130edb2d89a599
I20260806 12:56:54.696720 29703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513: Bootstrap starting.
I20260806 12:56:54.697265 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.695182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.24.83.62:35197)
0806 12:56:54.695409 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:54.695416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:54.695440 (+ 24us) server_negotiation.cc:407] Connection header received
0806 12:56:54.695831 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:54.695837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:54.695880 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:54.696078 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:54.696808 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:54.696811 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:54.696811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:54.696816 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:54.696880 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:54.696899 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:54.696996 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.697052 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.697109 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260806 12:56:54.697459 29703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:54.697788 29703 log.cc:824] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:54.698474 29703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513: No bootstrap required, opened a new log
I20260806 12:56:54.699849 29703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126afd6028fb483593351e35ace19513" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35197 } }
I20260806 12:56:54.699994 29703 raft_consensus.cc:374] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:54.700043 29703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 126afd6028fb483593351e35ace19513, State: Initialized, Role: FOLLOWER
I20260806 12:56:54.700167 29703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126afd6028fb483593351e35ace19513" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35197 } }
I20260806 12:56:54.700237 29703 raft_consensus.cc:388] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:54.700281 29703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:54.700341 29703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:54.700950 29703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126afd6028fb483593351e35ace19513" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35197 } }
I20260806 12:56:54.701061 29703 leader_election.cc:302] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 126afd6028fb483593351e35ace19513; no voters:
I20260806 12:56:54.701216 29703 leader_election.cc:288] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:54.701467 29708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:54.701488 29703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:54.701598 29708 raft_consensus.cc:686] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [term 1 LEADER]: Becoming Leader. State: Replica: 126afd6028fb483593351e35ace19513, State: Running, Role: LEADER
I20260806 12:56:54.701691 29708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126afd6028fb483593351e35ace19513" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35197 } }
I20260806 12:56:54.702072 29709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "126afd6028fb483593351e35ace19513" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126afd6028fb483593351e35ace19513" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35197 } } }
I20260806 12:56:54.702191 29709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:54.702401 29710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 126afd6028fb483593351e35ace19513. Latest consensus state: current_term: 1 leader_uuid: "126afd6028fb483593351e35ace19513" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "126afd6028fb483593351e35ace19513" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35197 } } }
I20260806 12:56:54.702414 29717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:54.702543 29710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:54.703063 29717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:54.704524 29717 catalog_manager.cc:1347] Generated new cluster ID: d402707c06114f24a2f68fdec7f6ba87
I20260806 12:56:54.704582 29717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:56:54.712626 29717 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:54.712776 29717 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:54.721033 29717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513: Generated new TSK 0
I20260806 12:56:54.721279 29717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:56:54.730506 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35197
--builtin_ntp_servers=127.24.83.20:37905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 12:56:54.811632 29727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:54.811820 29727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:54.811841 29727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:54.813022 29727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:54.813063 29727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:54.813089 29727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:56:54.814406 29727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:37905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:54.814636 29727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:54.814939 29727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:54.820777 29733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:54.820737 29735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:54.820717 29732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:54.821316 29727 server_base.cc:1034] running on GCE node
I20260806 12:56:54.821534 29727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:54.821779 29727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:54.822927 29727 hybrid_clock.cc:648] HybridClock initialized: now 1786021014822910 us; error 29 us; skew 500 ppm
I20260806 12:56:54.824513 29727 webserver.cc:460] Webserver started at http://127.24.83.1:36615/ using document root <none> and password file <none>
I20260806 12:56:54.824786 29727 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:54.824846 29727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:54.824970 29727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:54.826254 29727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "966798046bb6418a8ff04c23a15d01d9"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "d67fa43687c77c3d2898c366e3bc9492"
server_key_iv: "a407a6fb085455117e61079430bb4b1e"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.826658 29727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "966798046bb6418a8ff04c23a15d01d9"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "d67fa43687c77c3d2898c366e3bc9492"
server_key_iv: "a407a6fb085455117e61079430bb4b1e"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.828310 29727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 12:56:54.828956 29743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:54.829137 29727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:54.829231 29727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "966798046bb6418a8ff04c23a15d01d9"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "d67fa43687c77c3d2898c366e3bc9492"
server_key_iv: "a407a6fb085455117e61079430bb4b1e"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.829324 29727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:54.832532 29727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:54.832813 29727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:54.832963 29727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:54.833177 29727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:54.833487 29727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:54.833544 29727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:54.833582 29727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:54.833607 29727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:54.839329 29727 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:38723
I20260806 12:56:54.839365 29863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:38723 every 8 connection(s)
I20260806 12:56:54.839721 29727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:56:54.842634 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.840464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:40163 (local address 127.24.83.62:35197)
0806 12:56:54.840506 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:54.840508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:54.840742 (+ 234us) server_negotiation.cc:407] Connection header received
0806 12:56:54.841252 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:54.841253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:54.841260 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:54.841327 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:54.842394 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:54.842396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:54.842397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:54.842401 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:54.842416 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:54.842420 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:54.842476 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.842551 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.842565 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 12:56:54.842824 29866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.840428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35197 (local address 127.24.83.1:40163)
0806 12:56:54.840612 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:54.840653 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:54.841133 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:54.841395 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:54.841398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:54.841408 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:54.842247 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:56:54.842247 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:54.842273 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:56:54.842298 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:54.842483 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:54.842489 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:54.842492 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:56:54.842563 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 12:56:54.842621 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260806 12:56:54.843266 29864 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35197
I20260806 12:56:54.843336 29864 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:54.843492 29864 heartbeater.cc:511] Master 127.24.83.62:35197 requested a full tablet report, sending...
I20260806 12:56:54.843997 29666 ts_manager.cc:194] Registered new tserver with Master: 966798046bb6418a8ff04c23a15d01d9 (127.24.83.1:38723)
I20260806 12:56:54.845161 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 29727
I20260806 12:56:54.845278 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:56:54.845495 24908 external_mini_cluster.cc:1428] Setting key fc558e1caded561702b2e94cc996beb8
I20260806 12:56:54.846300 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35197
--builtin_ntp_servers=127.24.83.20:37905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 12:56:54.925186 29868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:54.925371 29868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:54.925393 29868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:54.926630 29868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:54.926671 29868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:54.926697 29868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:56:54.928131 29868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:37905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:54.928360 29868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:54.928638 29868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:54.933498 29873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:54.933491 29874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:54.933630 29868 server_base.cc:1034] running on GCE node
W20260806 12:56:54.933491 29876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:54.933981 29868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:54.934216 29868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:54.935355 29868 hybrid_clock.cc:648] HybridClock initialized: now 1786021014935344 us; error 27 us; skew 500 ppm
I20260806 12:56:54.940532 29868 webserver.cc:460] Webserver started at http://127.24.83.2:42203/ using document root <none> and password file <none>
I20260806 12:56:54.940784 29868 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:54.940832 29868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:54.940999 29868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:54.942057 29868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "27ab12a57df848018c5406339346b0e0"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "9ad48a8d5c2a6bacb7a10708a51f740a"
server_key_iv: "a2769cfafa3971adbeee50dd51c8c540"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.942420 29868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "27ab12a57df848018c5406339346b0e0"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "9ad48a8d5c2a6bacb7a10708a51f740a"
server_key_iv: "a2769cfafa3971adbeee50dd51c8c540"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.943732 29868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:54.944491 29882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:54.944674 29868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:54.944756 29868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "27ab12a57df848018c5406339346b0e0"
format_stamp: "Formatted at 2026-08-06 12:56:54 on dist-test-slave-zkpd"
server_key: "9ad48a8d5c2a6bacb7a10708a51f740a"
server_key_iv: "a2769cfafa3971adbeee50dd51c8c540"
server_key_version: "encryptionkey@0"
I20260806 12:56:54.944830 29868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:54.948127 29868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:54.948375 29868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:54.948508 29868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:54.948709 29868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:54.949000 29868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:54.949031 29868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:54.949095 29868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:54.949121 29868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:54.954764 29868 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:44379
I20260806 12:56:54.954799 30001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:44379 every 8 connection(s)
I20260806 12:56:54.955104 29868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:56:54.957932 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.955812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:48461 (local address 127.24.83.62:35197)
0806 12:56:54.955856 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:54.955858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:54.956130 (+ 272us) server_negotiation.cc:407] Connection header received
0806 12:56:54.956572 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:54.956573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:54.956577 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:54.956638 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:54.957725 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:54.957726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:54.957727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:54.957731 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:54.957746 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:54.957750 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:54.957816 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:54.957866 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 12:56:54.957878 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 12:56:54.958132 30004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:54.955782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35197 (local address 127.24.83.2:48461)
0806 12:56:54.955984 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:54.956033 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:54.956480 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:54.956706 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:54.956709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:54.956717 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:54.957567 (+ 850us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:56:54.957568 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:54.957594 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:56:54.957640 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:54.957818 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:54.957820 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:54.957823 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:56:54.957871 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 12:56:54.957915 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260806 12:56:54.958633 30002 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35197
I20260806 12:56:54.958719 30002 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:54.958834 30002 heartbeater.cc:511] Master 127.24.83.62:35197 requested a full tablet report, sending...
I20260806 12:56:54.959142 29666 ts_manager.cc:194] Registered new tserver with Master: 27ab12a57df848018c5406339346b0e0 (127.24.83.2:44379)
I20260806 12:56:54.960676 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 29868
I20260806 12:56:54.960757 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:56:54.960919 24908 external_mini_cluster.cc:1428] Setting key b0fea0a7760041869d8b2d228f355e20
I20260806 12:56:54.961829 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35197
--builtin_ntp_servers=127.24.83.20:37905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 12:56:55.041630 30005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:55.041810 30005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:55.041831 30005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:55.043004 30005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:55.043042 30005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:55.043093 30005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:56:55.044410 30005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:37905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:55.044642 30005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:55.044935 30005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:55.049907 30011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:55.049927 30010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:55.049926 30013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:55.050098 30005 server_base.cc:1034] running on GCE node
I20260806 12:56:55.050282 30005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:55.050560 30005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:55.051712 30005 hybrid_clock.cc:648] HybridClock initialized: now 1786021015051700 us; error 42 us; skew 500 ppm
I20260806 12:56:55.052868 30005 webserver.cc:460] Webserver started at http://127.24.83.3:39821/ using document root <none> and password file <none>
I20260806 12:56:55.053118 30005 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:55.053184 30005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:55.053318 30005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:55.054342 30005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "8a00944b79ce4d678218fa98c1a8247f"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "cb3d114d323c903c3a8fc14330b8cddb"
server_key_iv: "8c4d2ed530c94c28ca8c7bf6fdb7b5d7"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.054699 30005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "8a00944b79ce4d678218fa98c1a8247f"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "cb3d114d323c903c3a8fc14330b8cddb"
server_key_iv: "8c4d2ed530c94c28ca8c7bf6fdb7b5d7"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.055998 30005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:56:55.056712 30019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:55.056901 30005 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:55.056993 30005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "8a00944b79ce4d678218fa98c1a8247f"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "cb3d114d323c903c3a8fc14330b8cddb"
server_key_iv: "8c4d2ed530c94c28ca8c7bf6fdb7b5d7"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.057080 30005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:55.060352 30005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:55.060604 30005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:55.060739 30005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:55.060983 30005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:55.061281 30005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:55.061334 30005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:55.061372 30005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:55.061400 30005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:55.067019 30005 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:33025
I20260806 12:56:55.067072 30137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:33025 every 8 connection(s)
I20260806 12:56:55.067354 30005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:56:55.070281 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.068077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:33293 (local address 127.24.83.62:35197)
0806 12:56:55.068117 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.068119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.068363 (+ 244us) server_negotiation.cc:407] Connection header received
0806 12:56:55.068848 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.068849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.068858 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.068916 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:55.070011 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:55.070013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:55.070014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:55.070019 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:55.070035 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:55.070040 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:55.070096 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:55.070193 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 12:56:55.070210 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 12:56:55.070487 30140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.068076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35197 (local address 127.24.83.3:33293)
0806 12:56:55.068245 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:55.068275 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:55.068727 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:55.068981 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:55.068985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:55.068994 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:55.069863 (+ 869us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:56:55.069863 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:55.069889 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:56:55.069915 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:55.070104 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:55.070109 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:55.070112 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:56:55.070202 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 12:56:55.070261 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260806 12:56:55.070946 30138 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35197
I20260806 12:56:55.071041 30138 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:55.071179 30138 heartbeater.cc:511] Master 127.24.83.62:35197 requested a full tablet report, sending...
I20260806 12:56:55.071555 29666 ts_manager.cc:194] Registered new tserver with Master: 8a00944b79ce4d678218fa98c1a8247f (127.24.83.3:33025)
I20260806 12:56:55.076468 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 30005
I20260806 12:56:55.076571 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:56:55.076754 24908 external_mini_cluster.cc:1428] Setting key e1173b671816ba1610a5eb691a92e7f1
I20260806 12:56:55.077618 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:56:55.079973 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.078957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.24.83.62:35197)
0806 12:56:55.079005 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.079007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.079136 (+ 129us) server_negotiation.cc:407] Connection header received
0806 12:56:55.079180 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.079181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.079185 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.079233 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:55.079751 (+ 518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:55.079752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:55.079752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:55.079755 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:55.079767 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:55.079771 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:55.079824 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:55.079858 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 12:56:55.079870 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 12:56:55.081489 29666 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:56:55.083009 29666 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:56:55.086766 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.085216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:44379 (local address 127.0.0.1:45842)
0806 12:56:55.085445 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:55.085453 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:55.085491 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:55.085772 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:55.085772 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:55.085775 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:55.086317 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:56:55.086318 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:55.086334 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:56:55.086358 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:55.086577 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:55.086585 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:55.086588 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:56:55.086674 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 12:56:55.086688 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":76,"threads_started":1}
I20260806 12:56:55.086798 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.085432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45842 (local address 127.24.83.2:44379)
0806 12:56:55.085601 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.085605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.085615 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:56:55.085640 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.085642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.085657 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.085755 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:55.086448 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:55.086456 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:55.086457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:55.086462 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:55.086494 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:55.086519 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:55.086576 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:55.086687 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 12:56:55.086704 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260806 12:56:55.087240 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.085019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:38723 (local address 127.0.0.1:42376)
0806 12:56:55.085202 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:55.085257 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:55.085316 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:55.085744 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:55.085751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:55.085758 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:55.086656 (+ 898us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:56:55.086657 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:55.086664 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:56:55.086677 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:55.087095 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:55.087096 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:55.087096 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 12:56:55.087151 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 12:56:55.087182 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":59,"threads_started":1}
I20260806 12:56:55.087280 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.085117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42376 (local address 127.24.83.1:38723)
0806 12:56:55.085604 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.085608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.085616 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:56:55.085637 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.085639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.085646 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.085702 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:55.086911 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:55.086924 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:55.086924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:55.086930 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:55.086967 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:55.087000 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:55.087097 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:55.087179 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 12:56:55.087195 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":88,"threads_started":1}
I20260806 12:56:55.087373 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.085047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:33025 (local address 127.0.0.1:41042)
0806 12:56:55.085310 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:55.085321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:55.085371 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:55.086261 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:55.086262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:55.086265 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:55.086656 (+ 391us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:56:55.086657 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:55.086667 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:56:55.086680 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:55.087152 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:55.087172 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:55.087172 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 12:56:55.087216 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 12:56:55.087230 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":99,"threads_started":1}
I20260806 12:56:55.087615 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.085208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.24.83.3:33025)
0806 12:56:55.086108 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.086111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.086118 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:56:55.086136 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.086137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.086143 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.086237 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:55.086760 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:55.086766 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:55.086766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:55.086770 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:55.086806 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:55.087072 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:55.087128 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:55.087276 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 12:56:55.087536 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":105,"threads_started":1}
I20260806 12:56:55.087670 29934 tablet_service.cc:1467] Processing CreateTablet for tablet 5491bbd8b32a426ab0003e252051ad90 (DEFAULT_TABLE table=test-table [id=848716a1705c4a188f7ac766ebf2362d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:55.088037 29934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5491bbd8b32a426ab0003e252051ad90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:55.088236 30070 tablet_service.cc:1467] Processing CreateTablet for tablet 5491bbd8b32a426ab0003e252051ad90 (DEFAULT_TABLE table=test-table [id=848716a1705c4a188f7ac766ebf2362d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:55.088546 30070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5491bbd8b32a426ab0003e252051ad90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:55.090411 30152 tablet_bootstrap.cc:492] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0: Bootstrap starting.
I20260806 12:56:55.091051 30152 tablet_bootstrap.cc:654] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:55.091018 29796 tablet_service.cc:1467] Processing CreateTablet for tablet 5491bbd8b32a426ab0003e252051ad90 (DEFAULT_TABLE table=test-table [id=848716a1705c4a188f7ac766ebf2362d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:56:55.091236 30153 tablet_bootstrap.cc:492] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f: Bootstrap starting.
I20260806 12:56:55.091364 29796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5491bbd8b32a426ab0003e252051ad90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:55.091375 30152 log.cc:824] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:55.091820 30153 tablet_bootstrap.cc:654] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:55.092206 30153 log.cc:824] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:55.092238 30152 tablet_bootstrap.cc:492] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0: No bootstrap required, opened a new log
I20260806 12:56:55.092293 30152 ts_tablet_manager.cc:1397] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:56:55.092978 30153 tablet_bootstrap.cc:492] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f: No bootstrap required, opened a new log
I20260806 12:56:55.093046 30153 ts_tablet_manager.cc:1397] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:56:55.094116 30156 tablet_bootstrap.cc:492] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9: Bootstrap starting.
I20260806 12:56:55.094780 30156 tablet_bootstrap.cc:654] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:55.094877 30152 raft_consensus.cc:348] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "966798046bb6418a8ff04c23a15d01d9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38723 } } peers { permanent_uuid: "8a00944b79ce4d678218fa98c1a8247f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 33025 } } peers { permanent_uuid: "27ab12a57df848018c5406339346b0e0" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44379 } }
I20260806 12:56:55.095064 30152 raft_consensus.cc:374] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:55.095111 30152 raft_consensus.cc:729] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27ab12a57df848018c5406339346b0e0, State: Initialized, Role: FOLLOWER
I20260806 12:56:55.095114 30156 log.cc:824] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:55.095283 30152 consensus_queue.cc:260] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "966798046bb6418a8ff04c23a15d01d9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38723 } } peers { permanent_uuid: "8a00944b79ce4d678218fa98c1a8247f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 33025 } } peers { permanent_uuid: "27ab12a57df848018c5406339346b0e0" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44379 } }
I20260806 12:56:55.095587 30152 ts_tablet_manager.cc:1428] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260806 12:56:55.095606 30002 heartbeater.cc:503] Master 127.24.83.62:35197 was elected leader, sending a full tablet report...
I20260806 12:56:55.095824 30156 tablet_bootstrap.cc:492] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9: No bootstrap required, opened a new log
I20260806 12:56:55.095887 30156 ts_tablet_manager.cc:1397] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:56:55.095800 30153 raft_consensus.cc:348] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "966798046bb6418a8ff04c23a15d01d9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38723 } } peers { permanent_uuid: "8a00944b79ce4d678218fa98c1a8247f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 33025 } } peers { permanent_uuid: "27ab12a57df848018c5406339346b0e0" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44379 } }
I20260806 12:56:55.096002 30153 raft_consensus.cc:374] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:55.096051 30153 raft_consensus.cc:729] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a00944b79ce4d678218fa98c1a8247f, State: Initialized, Role: FOLLOWER
I20260806 12:56:55.096156 30153 consensus_queue.cc:260] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "966798046bb6418a8ff04c23a15d01d9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38723 } } peers { permanent_uuid: "8a00944b79ce4d678218fa98c1a8247f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 33025 } } peers { permanent_uuid: "27ab12a57df848018c5406339346b0e0" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44379 } }
I20260806 12:56:55.096403 30153 ts_tablet_manager.cc:1428] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 12:56:55.096423 30138 heartbeater.cc:503] Master 127.24.83.62:35197 was elected leader, sending a full tablet report...
I20260806 12:56:55.098111 30156 raft_consensus.cc:348] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "966798046bb6418a8ff04c23a15d01d9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38723 } } peers { permanent_uuid: "8a00944b79ce4d678218fa98c1a8247f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 33025 } } peers { permanent_uuid: "27ab12a57df848018c5406339346b0e0" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44379 } }
I20260806 12:56:55.098261 30156 raft_consensus.cc:374] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:55.098315 30156 raft_consensus.cc:729] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 966798046bb6418a8ff04c23a15d01d9, State: Initialized, Role: FOLLOWER
I20260806 12:56:55.098428 30156 consensus_queue.cc:260] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "966798046bb6418a8ff04c23a15d01d9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38723 } } peers { permanent_uuid: "8a00944b79ce4d678218fa98c1a8247f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 33025 } } peers { permanent_uuid: "27ab12a57df848018c5406339346b0e0" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44379 } }
I20260806 12:56:55.098654 30156 ts_tablet_manager.cc:1428] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 12:56:55.098718 29864 heartbeater.cc:503] Master 127.24.83.62:35197 was elected leader, sending a full tablet report...
I20260806 12:56:55.156724 30158 raft_consensus.cc:482] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:56:55.156847 30158 raft_consensus.cc:504] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "966798046bb6418a8ff04c23a15d01d9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38723 } } peers { permanent_uuid: "8a00944b79ce4d678218fa98c1a8247f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 33025 } } peers { permanent_uuid: "27ab12a57df848018c5406339346b0e0" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44379 } }
I20260806 12:56:55.157266 30158 leader_election.cc:288] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 966798046bb6418a8ff04c23a15d01d9 (127.24.83.1:38723), 8a00944b79ce4d678218fa98c1a8247f (127.24.83.3:33025)
I20260806 12:56:55.158612 30004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.157438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:33025 (local address 127.24.83.2:37759)
0806 12:56:55.157495 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:55.157506 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:55.157546 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:55.157719 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:55.157720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:55.157723 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:55.158226 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:56:55.158227 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:55.158236 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:56:55.158250 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:55.158453 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:55.158453 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:55.158454 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 12:56:55.158525 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 12:56:55.158542 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 12:56:55.158612 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.157528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:37759 (local address 127.24.83.3:33025)
0806 12:56:55.157591 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.157593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.157603 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:56:55.157630 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.157631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.157635 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.157702 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:55.158335 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:55.158336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:55.158337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:55.158341 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:55.158378 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:55.158383 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:55.158448 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:55.158530 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 12:56:55.158546 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:56:55.158736 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.157527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:32921 (local address 127.24.83.1:38723)
0806 12:56:55.157587 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.157589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.157707 (+ 118us) server_negotiation.cc:407] Connection header received
0806 12:56:55.157764 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.157766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.157771 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.157853 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:55.158505 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:55.158506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:55.158507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:55.158511 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:55.158525 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:55.158530 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:55.158606 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:55.158654 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 12:56:55.158669 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:56:55.158748 30161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.157466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:38723 (local address 127.24.83.2:32921)
0806 12:56:55.157645 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:55.157652 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:55.157709 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:55.157858 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:55.157859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:55.157861 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:55.158419 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:56:55.158420 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:55.158426 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:56:55.158439 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:55.158606 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:55.158607 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:55.158608 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 12:56:55.158657 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 12:56:55.158669 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":117,"threads_started":1}
I20260806 12:56:55.158951 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5491bbd8b32a426ab0003e252051ad90" candidate_uuid: "27ab12a57df848018c5406339346b0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a00944b79ce4d678218fa98c1a8247f" is_pre_election: true
I20260806 12:56:55.159068 30091 raft_consensus.cc:2454] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27ab12a57df848018c5406339346b0e0 in term 0.
I20260806 12:56:55.159312 29884 leader_election.cc:302] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27ab12a57df848018c5406339346b0e0, 8a00944b79ce4d678218fa98c1a8247f; no voters:
I20260806 12:56:55.159401 29816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5491bbd8b32a426ab0003e252051ad90" candidate_uuid: "27ab12a57df848018c5406339346b0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "966798046bb6418a8ff04c23a15d01d9" is_pre_election: true
I20260806 12:56:55.159559 29816 raft_consensus.cc:2454] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27ab12a57df848018c5406339346b0e0 in term 0.
I20260806 12:56:55.159703 30158 raft_consensus.cc:2781] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:56:55.159770 30158 raft_consensus.cc:482] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:56:55.159803 30158 raft_consensus.cc:3037] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:55.160661 30158 raft_consensus.cc:504] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "966798046bb6418a8ff04c23a15d01d9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38723 } } peers { permanent_uuid: "8a00944b79ce4d678218fa98c1a8247f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 33025 } } peers { permanent_uuid: "27ab12a57df848018c5406339346b0e0" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44379 } }
I20260806 12:56:55.160807 30158 leader_election.cc:288] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [CANDIDATE]: Term 1 election: Requested vote from peers 966798046bb6418a8ff04c23a15d01d9 (127.24.83.1:38723), 8a00944b79ce4d678218fa98c1a8247f (127.24.83.3:33025)
I20260806 12:56:55.160957 29816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5491bbd8b32a426ab0003e252051ad90" candidate_uuid: "27ab12a57df848018c5406339346b0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "966798046bb6418a8ff04c23a15d01d9"
I20260806 12:56:55.160981 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5491bbd8b32a426ab0003e252051ad90" candidate_uuid: "27ab12a57df848018c5406339346b0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a00944b79ce4d678218fa98c1a8247f"
I20260806 12:56:55.161047 30091 raft_consensus.cc:3037] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:55.161053 29816 raft_consensus.cc:3037] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:55.161701 30091 raft_consensus.cc:2454] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27ab12a57df848018c5406339346b0e0 in term 1.
I20260806 12:56:55.161830 29816 raft_consensus.cc:2454] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27ab12a57df848018c5406339346b0e0 in term 1.
I20260806 12:56:55.161857 29884 leader_election.cc:302] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27ab12a57df848018c5406339346b0e0, 8a00944b79ce4d678218fa98c1a8247f; no voters:
I20260806 12:56:55.161957 30158 raft_consensus.cc:2781] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:55.162135 30158 raft_consensus.cc:686] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 1 LEADER]: Becoming Leader. State: Replica: 27ab12a57df848018c5406339346b0e0, State: Running, Role: LEADER
I20260806 12:56:55.162253 30158 consensus_queue.cc:237] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "966798046bb6418a8ff04c23a15d01d9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38723 } } peers { permanent_uuid: "8a00944b79ce4d678218fa98c1a8247f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 33025 } } peers { permanent_uuid: "27ab12a57df848018c5406339346b0e0" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44379 } }
I20260806 12:56:55.163048 29665 catalog_manager.cc:5495] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27ab12a57df848018c5406339346b0e0 (127.24.83.2). New cstate: current_term: 1 leader_uuid: "27ab12a57df848018c5406339346b0e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "966798046bb6418a8ff04c23a15d01d9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a00944b79ce4d678218fa98c1a8247f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 33025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27ab12a57df848018c5406339346b0e0" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44379 } health_report { overall_health: HEALTHY } } }
I20260806 12:56:55.183844 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.182847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45856 (local address 127.24.83.2:44379)
0806 12:56:55.182914 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.182916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.182927 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:56:55.183008 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.183008 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.183021 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.183087 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:55.183608 (+ 521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:55.183609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:55.183609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:55.183614 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:55.183628 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:55.183632 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:55.183688 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:55.183749 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 12:56:55.183763 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 12:56:55.185662 29816 raft_consensus.cc:1261] T 5491bbd8b32a426ab0003e252051ad90 P 966798046bb6418a8ff04c23a15d01d9 [term 1 FOLLOWER]: Refusing update from remote peer 27ab12a57df848018c5406339346b0e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:55.185662 30091 raft_consensus.cc:1261] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f [term 1 FOLLOWER]: Refusing update from remote peer 27ab12a57df848018c5406339346b0e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:56:55.186020 30158 consensus_queue.cc:1035] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a00944b79ce4d678218fa98c1a8247f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 33025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:55.186146 30162 consensus_queue.cc:1035] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "966798046bb6418a8ff04c23a15d01d9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:56:55.188428 30165 mvcc.cc:205] Tried to move back new op lower bound from 7315542078197837824 to 7315542078105239552. Current Snapshot: MvccSnapshot[applied={T|T < 7315542078197837824}]
I20260806 12:56:55.188907 30166 mvcc.cc:205] Tried to move back new op lower bound from 7315542078197837824 to 7315542078105239552. Current Snapshot: MvccSnapshot[applied={T|T < 7315542078197837824}]
I20260806 12:56:55.189218 30167 mvcc.cc:205] Tried to move back new op lower bound from 7315542078197837824 to 7315542078105239552. Current Snapshot: MvccSnapshot[applied={T|T < 7315542078197837824}]
I20260806 12:56:55.190898 29665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:55.190985 29665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:56:55.191720 29665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 126afd6028fb483593351e35ace19513: Sending DeleteTablet for 3 replicas of tablet 5491bbd8b32a426ab0003e252051ad90
I20260806 12:56:55.192075 29796 tablet_service.cc:1514] Processing DeleteTablet for tablet 5491bbd8b32a426ab0003e252051ad90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:55 UTC) from {username='slave'} at 127.0.0.1:42376
I20260806 12:56:55.192085 30070 tablet_service.cc:1514] Processing DeleteTablet for tablet 5491bbd8b32a426ab0003e252051ad90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:55 UTC) from {username='slave'} at 127.0.0.1:41042
I20260806 12:56:55.192144 29934 tablet_service.cc:1514] Processing DeleteTablet for tablet 5491bbd8b32a426ab0003e252051ad90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:56:55 UTC) from {username='slave'} at 127.0.0.1:45842
I20260806 12:56:55.192332 30177 tablet_replica.cc:330] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0: stopping tablet replica
I20260806 12:56:55.192427 30177 raft_consensus.cc:2229] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:56:55.192528 30177 raft_consensus.cc:2258] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:55.192595 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 29727
I20260806 12:56:55.193866 30176 tablet_replica.cc:330] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f: stopping tablet replica
I20260806 12:56:55.193977 30176 raft_consensus.cc:2229] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:56:55.194074 30176 raft_consensus.cc:2258] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:56:55.194600 30177 ts_tablet_manager.cc:1905] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:55.196203 30176 ts_tablet_manager.cc:1905] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:56:55.196640 30177 ts_tablet_manager.cc:1918] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:56:55.196719 30177 log.cc:1196] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/5491bbd8b32a426ab0003e252051ad90
I20260806 12:56:55.197007 30177 ts_tablet_manager.cc:1939] T 5491bbd8b32a426ab0003e252051ad90 P 27ab12a57df848018c5406339346b0e0: Deleting consensus metadata
I20260806 12:56:55.197579 29652 catalog_manager.cc:4841] TS 27ab12a57df848018c5406339346b0e0 (127.24.83.2:44379): tablet 5491bbd8b32a426ab0003e252051ad90 (table test-table [id=848716a1705c4a188f7ac766ebf2362d]) successfully deleted
I20260806 12:56:55.198213 30176 ts_tablet_manager.cc:1918] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:56:55.198305 30176 log.cc:1196] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/5491bbd8b32a426ab0003e252051ad90
I20260806 12:56:55.198568 30176 ts_tablet_manager.cc:1939] T 5491bbd8b32a426ab0003e252051ad90 P 8a00944b79ce4d678218fa98c1a8247f: Deleting consensus metadata
I20260806 12:56:55.198997 29651 catalog_manager.cc:4841] TS 8a00944b79ce4d678218fa98c1a8247f (127.24.83.3:33025): tablet 5491bbd8b32a426ab0003e252051ad90 (table test-table [id=848716a1705c4a188f7ac766ebf2362d]) successfully deleted
W20260806 12:56:55.199864 29652 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:38723 (error 108)
I20260806 12:56:55.200145 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 29868
I20260806 12:56:55.201077 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.200338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:38723 (local address 127.0.0.1:42390)
0806 12:56:55.200985 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:55.201038 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:38723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":623}
W20260806 12:56:55.201504 29652 catalog_manager.cc:4584] TS 966798046bb6418a8ff04c23a15d01d9 (127.24.83.1:38723): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5491bbd8b32a426ab0003e252051ad90: Network error: Client connection negotiation failed: client connection to 127.24.83.1:38723: connect: Connection refused (error 111)
I20260806 12:56:55.206080 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 30005
I20260806 12:56:55.214211 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 29634
2026-08-06T12:56:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (651 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 12:56:55.234763 24908 mini_oidc.cc:152] Starting JWKS server
I20260806 12:56:55.235858 24908 webserver.cc:460] Webserver started at https://127.0.0.1:35739/ using document root <none> and password file <none>
I20260806 12:56:55.235991 24908 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 12:56:55.236133 24908 webserver.cc:460] Webserver started at http://127.0.0.1:39351/ using document root <none> and password file <none>
2026-08-06T12:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:55Z Disabled control of system clock
I20260806 12:56:55.244021 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40989
--webserver_interface=127.24.83.62
--webserver_port=0
--builtin_ntp_servers=127.24.83.20:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:40989
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35739/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/testchainca.cert with env {}
W20260806 12:56:55.324124 30184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:55.324312 30184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:55.324333 30184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:55.325479 30184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:55.325519 30184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:55.325532 30184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:55.325544 30184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:55.325555 30184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 12:56:55.325567 30184 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 12:56:55.325578 30184 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35739/jwks.json
I20260806 12:56:55.327111 30184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:40989
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35739/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:55.327363 30184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:55.327582 30184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:55.332227 30190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:55.332399 30184 server_base.cc:1034] running on GCE node
W20260806 12:56:55.332284 30192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:55.332227 30189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:55.333528 30184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:55.333789 30184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:55.334950 30184 hybrid_clock.cc:648] HybridClock initialized: now 1786021015334942 us; error 36 us; skew 500 ppm
I20260806 12:56:55.336205 30184 webserver.cc:460] Webserver started at http://127.24.83.62:46369/ using document root <none> and password file <none>
I20260806 12:56:55.336452 30184 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:55.336520 30184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:55.336669 30184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:55.337685 30184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "391b69f99a2c47eb93b416eaaa7b2b12"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "1662853ad96dcb61943f1446362d8120"
server_key_iv: "dcb481adb133aca9cadf6e2da7a4e589"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.338073 30184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "391b69f99a2c47eb93b416eaaa7b2b12"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "1662853ad96dcb61943f1446362d8120"
server_key_iv: "dcb481adb133aca9cadf6e2da7a4e589"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.339255 30184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:56:55.339952 30198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:55.340147 30184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:56:55.340219 30184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "391b69f99a2c47eb93b416eaaa7b2b12"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "1662853ad96dcb61943f1446362d8120"
server_key_iv: "dcb481adb133aca9cadf6e2da7a4e589"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.340268 30184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:55.349704 30184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:55.349925 30184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:55.350019 30184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:55.353585 30184 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:40989
I20260806 12:56:55.353649 30254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:40989 every 8 connection(s)
I20260806 12:56:55.353967 30184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:55.354894 30255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:55.357139 30255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12: Bootstrap starting.
I20260806 12:56:55.357722 30255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:55.358017 30255 log.cc:824] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:55.358196 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 30184
I20260806 12:56:55.358314 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:55.358523 24908 external_mini_cluster.cc:1428] Setting key 3c48af10f347e14bbe153e6c1c07ab0a
I20260806 12:56:55.358774 30255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12: No bootstrap required, opened a new log
I20260806 12:56:55.361600 30255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391b69f99a2c47eb93b416eaaa7b2b12" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40989 } }
I20260806 12:56:55.361752 30255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:55.361778 30255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 391b69f99a2c47eb93b416eaaa7b2b12, State: Initialized, Role: FOLLOWER
I20260806 12:56:55.361881 30255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391b69f99a2c47eb93b416eaaa7b2b12" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40989 } }
I20260806 12:56:55.361964 30255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:55.362000 30255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:55.362061 30255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:55.362600 30255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391b69f99a2c47eb93b416eaaa7b2b12" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40989 } }
I20260806 12:56:55.362684 30255 leader_election.cc:302] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 391b69f99a2c47eb93b416eaaa7b2b12; no voters:
I20260806 12:56:55.362857 30255 leader_election.cc:288] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:55.362881 30260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:55.362995 30260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [term 1 LEADER]: Becoming Leader. State: Replica: 391b69f99a2c47eb93b416eaaa7b2b12, State: Running, Role: LEADER
I20260806 12:56:55.363085 30260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391b69f99a2c47eb93b416eaaa7b2b12" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40989 } }
I20260806 12:56:55.363140 30255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:55.363443 30261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "391b69f99a2c47eb93b416eaaa7b2b12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391b69f99a2c47eb93b416eaaa7b2b12" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40989 } } }
I20260806 12:56:55.363457 30262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 391b69f99a2c47eb93b416eaaa7b2b12. Latest consensus state: current_term: 1 leader_uuid: "391b69f99a2c47eb93b416eaaa7b2b12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391b69f99a2c47eb93b416eaaa7b2b12" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40989 } } }
I20260806 12:56:55.363535 30261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:55.363564 30262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:55.363773 30266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:55.364331 30266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:55.365734 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.359042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54574 (local address 127.24.83.62:40989)
0806 12:56:55.359272 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.359281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.359314 (+ 33us) server_negotiation.cc:407] Connection header received
0806 12:56:55.360151 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.360160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.360260 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.360409 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:55.360569 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.361968 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.362972 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.363821 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.364913 (+ 1092us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:55.365060 (+ 147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:55.365063 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:55.365063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:55.365257 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:55.365293 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:55.365305 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:55.365369 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:55.365449 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 12:56:55.365525 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260806 12:56:55.366653 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:56:55.368806 30266 catalog_manager.cc:1347] Generated new cluster ID: e6b3c0557851474e9c6eb836c00b5e1c
I20260806 12:56:55.368860 30266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:56:55.378638 30266 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:55.379288 30266 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:55.387336 30266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 391b69f99a2c47eb93b416eaaa7b2b12: Generated new TSK 0
I20260806 12:56:55.387516 30266 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 12:56:55.414714 30285 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 12:56:55.414883 30285 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 12:56:55.415084 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.399067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54584 (local address 127.24.83.62:40989)
0806 12:56:55.399112 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.399114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.399272 (+ 158us) server_negotiation.cc:407] Connection header received
0806 12:56:55.399377 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.399378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.399389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.399440 (+ 51us) server_negotiation.cc:226] Negotiated authn=JWT
0806 12:56:55.399583 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.400126 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.401265 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.401463 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.401516 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:55.401743 (+ 227us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 12:56:55.414839 (+ 13096us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 12:56:55.415014 (+ 175us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54584: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 12:56:55.415181 30259 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54584: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 12:56:55.415552 30284 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:56:55.398955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40989 (local address 127.0.0.1:54584)
0806 12:56:55.399210 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:55.399221 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:55.399292 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:55.399455 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:55.399455 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:55.399457 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 12:56:55.399527 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:55.399529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.400193 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.400194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:55.401180 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:55.401183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.401545 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.401546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:55.401618 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:55.401641 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 12:56:55.415036 (+ 13395us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 12:56:55.415161 (+ 125us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.83.62:40989: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":117,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.83.62:40989: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 12:56:55.416178 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 30184
2026-08-06T12:56:55Z chronyd exiting
I20260806 12:56:55.452983 24908 test_util.cc:175] -----------------------------------------------
I20260806 12:56:55.453055 24908 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786020991335182-24908-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 12:56:55.453857 24908 mini_oidc.cc:152] Starting JWKS server
I20260806 12:56:55.454656 24908 webserver.cc:460] Webserver started at https://127.0.0.1:44225/ using document root <none> and password file <none>
I20260806 12:56:55.454731 24908 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 12:56:55.454967 24908 webserver.cc:460] Webserver started at http://127.0.0.1:37899/ using document root <none> and password file <none>
2026-08-06T12:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:55Z Disabled control of system clock
I20260806 12:56:55.464188 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:46341
--webserver_interface=127.24.83.62
--webserver_port=0
--builtin_ntp_servers=127.24.83.20:45827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:46341
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44225/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/testchainca.cert with env {}
W20260806 12:56:55.544045 30291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:55.544239 30291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:55.544261 30291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:55.545344 30291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:55.545377 30291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:55.545389 30291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:55.545401 30291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:55.545413 30291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 12:56:55.545424 30291 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 12:56:55.545435 30291 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44225/jwks.json
I20260806 12:56:55.546964 30291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:45827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:46341
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:46341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44225/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:55.547155 30291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:55.547310 30291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:55.551980 30296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:55.552044 30297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:55.552054 30299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:55.552157 30291 server_base.cc:1034] running on GCE node
I20260806 12:56:55.552372 30291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:55.552649 30291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:55.553833 30291 hybrid_clock.cc:648] HybridClock initialized: now 1786021015553821 us; error 27 us; skew 500 ppm
I20260806 12:56:55.555105 30291 webserver.cc:460] Webserver started at http://127.24.83.62:39917/ using document root <none> and password file <none>
I20260806 12:56:55.555362 30291 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:55.555431 30291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:55.555569 30291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:55.556641 30291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "e5d8fdabfb95422681724bede3157a55"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "ae1376b7c4da1a3205bcb38c780cc075"
server_key_iv: "d82e21973a6e76c4cc1ebbc302e19208"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.556996 30291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "e5d8fdabfb95422681724bede3157a55"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "ae1376b7c4da1a3205bcb38c780cc075"
server_key_iv: "d82e21973a6e76c4cc1ebbc302e19208"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.558409 30291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 12:56:55.559262 30305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:55.559463 30291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:56:55.559566 30291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "e5d8fdabfb95422681724bede3157a55"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "ae1376b7c4da1a3205bcb38c780cc075"
server_key_iv: "d82e21973a6e76c4cc1ebbc302e19208"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.559655 30291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:55.566964 30291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:55.567232 30291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:55.567382 30291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:55.571359 30291 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:46341
I20260806 12:56:55.571393 30359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:46341 every 8 connection(s)
I20260806 12:56:55.571699 30291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:55.572362 30360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:55.574949 30360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55: Bootstrap starting.
I20260806 12:56:55.575448 30360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:55.575671 30360 log.cc:824] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:55.576327 30360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55: No bootstrap required, opened a new log
I20260806 12:56:55.577800 30360 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d8fdabfb95422681724bede3157a55" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46341 } }
I20260806 12:56:55.577931 30360 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:55.577953 30360 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5d8fdabfb95422681724bede3157a55, State: Initialized, Role: FOLLOWER
I20260806 12:56:55.578047 30360 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d8fdabfb95422681724bede3157a55" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46341 } }
I20260806 12:56:55.578095 30360 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:55.578115 30360 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:55.578140 30360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:55.578600 30360 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d8fdabfb95422681724bede3157a55" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46341 } }
I20260806 12:56:55.578653 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 30291
I20260806 12:56:55.578720 30360 leader_election.cc:302] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5d8fdabfb95422681724bede3157a55; no voters:
I20260806 12:56:55.578765 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:55.578933 30360 leader_election.cc:288] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:55.578964 30363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:55.578961 24908 external_mini_cluster.cc:1428] Setting key 84395c9deef030182f9699a65226ea5f
I20260806 12:56:55.579087 30363 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [term 1 LEADER]: Becoming Leader. State: Replica: e5d8fdabfb95422681724bede3157a55, State: Running, Role: LEADER
I20260806 12:56:55.579195 30360 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:55.579175 30363 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d8fdabfb95422681724bede3157a55" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46341 } }
I20260806 12:56:55.579550 30363 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5d8fdabfb95422681724bede3157a55. Latest consensus state: current_term: 1 leader_uuid: "e5d8fdabfb95422681724bede3157a55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d8fdabfb95422681724bede3157a55" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46341 } } }
I20260806 12:56:55.579640 30363 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:55.579559 30364 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5d8fdabfb95422681724bede3157a55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5d8fdabfb95422681724bede3157a55" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46341 } } }
I20260806 12:56:55.579797 30364 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:55.580034 30371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:55.580523 30371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:55.582131 30371 catalog_manager.cc:1347] Generated new cluster ID: cbe3adce0f3c42f0abbf964280b630f1
I20260806 12:56:55.582181 30371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:56:55.585516 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.579579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46298 (local address 127.24.83.62:46341)
0806 12:56:55.579813 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.579822 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.580062 (+ 240us) server_negotiation.cc:407] Connection header received
0806 12:56:55.580333 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.580341 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.580407 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.580520 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:55.581033 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.582273 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.583398 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.584031 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.584141 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:55.584824 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:55.584828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:55.584828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:55.585016 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:55.585053 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:55.585066 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:55.585130 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:55.585228 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 12:56:55.585304 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260806 12:56:55.586403 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:56:55.596392 30371 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:55.597074 30371 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:55.603018 30371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5d8fdabfb95422681724bede3157a55: Generated new TSK 0
I20260806 12:56:55.603190 30371 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 12:56:55.622905 30392 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 12:56:55.622982 30392 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 12:56:55.623364 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.609354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46310 (local address 127.24.83.62:46341)
0806 12:56:55.609415 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.609417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.609530 (+ 113us) server_negotiation.cc:407] Connection header received
0806 12:56:55.609635 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.609636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.609648 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.609706 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0806 12:56:55.609857 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.610341 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.611133 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.611360 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.611438 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:55.611635 (+ 197us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 12:56:55.623113 (+ 11478us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44225/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 12:56:55.623281 (+ 168us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46310: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44225/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 12:56:55.623421 30369 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46310: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44225/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 12:56:55.623442 30391 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:56:55.609278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:46341 (local address 127.0.0.1:46310)
0806 12:56:55.609469 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:55.609477 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:55.609547 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:55.609715 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:55.609716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:55.609717 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 12:56:55.609774 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:55.609775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.610429 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.610430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:55.611027 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:55.611029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.611440 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.611441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:55.611510 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:55.611522 (+ 12us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 12:56:55.623233 (+ 11711us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44225/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 12:56:55.623338 (+ 105us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.83.62:46341: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44225/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":111,"threads_started":1}
I20260806 12:56:55.623724 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 30291
2026-08-06T12:56:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (208 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 12:56:55.661921 24908 mini_oidc.cc:152] Starting JWKS server
I20260806 12:56:55.662567 24908 webserver.cc:460] Webserver started at https://127.0.0.1:43315/ using document root <none> and password file <none>
I20260806 12:56:55.662606 24908 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 12:56:55.662740 24908 webserver.cc:460] Webserver started at http://127.0.0.1:45893/ using document root <none> and password file <none>
2026-08-06T12:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:55Z Disabled control of system clock
I20260806 12:56:55.670316 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45541
--webserver_interface=127.24.83.62
--webserver_port=0
--builtin_ntp_servers=127.24.83.20:36295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:45541
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43315/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 12:56:55.749320 30398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:55.749539 30398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:55.749572 30398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:55.750723 30398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:55.750792 30398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:55.750813 30398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:55.750834 30398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:55.750862 30398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 12:56:55.750885 30398 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 12:56:55.750906 30398 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43315/jwks.json
I20260806 12:56:55.752314 30398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:45541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43315/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:55.752575 30398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:55.752838 30398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:55.757740 30403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:55.757814 30404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:55.757726 30406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:55.757833 30398 server_base.cc:1034] running on GCE node
I20260806 12:56:55.758145 30398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:55.758414 30398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:55.759552 30398 hybrid_clock.cc:648] HybridClock initialized: now 1786021015759539 us; error 28 us; skew 500 ppm
I20260806 12:56:55.760764 30398 webserver.cc:460] Webserver started at http://127.24.83.62:45255/ using document root <none> and password file <none>
I20260806 12:56:55.761026 30398 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:55.761098 30398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:55.761250 30398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:55.762262 30398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "bea5d6a7b8f54d91aa2309e8594b5a52"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "7231362065b0f245e39ad1e08c0fc1fd"
server_key_iv: "32f71285be99a4ec0f7e3b2b2cea6093"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.762614 30398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "bea5d6a7b8f54d91aa2309e8594b5a52"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "7231362065b0f245e39ad1e08c0fc1fd"
server_key_iv: "32f71285be99a4ec0f7e3b2b2cea6093"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.763931 30398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:56:55.764674 30412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:55.764860 30398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:55.764961 30398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "bea5d6a7b8f54d91aa2309e8594b5a52"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "7231362065b0f245e39ad1e08c0fc1fd"
server_key_iv: "32f71285be99a4ec0f7e3b2b2cea6093"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.765045 30398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:55.781143 30398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:55.781431 30398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:55.781577 30398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:55.785550 30398 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:45541
I20260806 12:56:55.785586 30464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:45541 every 8 connection(s)
I20260806 12:56:55.785934 30398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:55.786573 30465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:55.789327 30465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52: Bootstrap starting.
I20260806 12:56:55.789888 30465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:55.790175 30465 log.cc:824] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:55.790828 30465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52: No bootstrap required, opened a new log
I20260806 12:56:55.792421 30465 raft_consensus.cc:348] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5d6a7b8f54d91aa2309e8594b5a52" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45541 } }
I20260806 12:56:55.792552 30465 raft_consensus.cc:374] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:55.792599 30465 raft_consensus.cc:729] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea5d6a7b8f54d91aa2309e8594b5a52, State: Initialized, Role: FOLLOWER
I20260806 12:56:55.792761 30465 consensus_queue.cc:260] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5d6a7b8f54d91aa2309e8594b5a52" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45541 } }
I20260806 12:56:55.792847 30465 raft_consensus.cc:388] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:55.792883 30465 raft_consensus.cc:482] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:55.792939 30465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:55.793462 30465 raft_consensus.cc:504] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5d6a7b8f54d91aa2309e8594b5a52" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45541 } }
I20260806 12:56:55.793548 30465 leader_election.cc:302] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bea5d6a7b8f54d91aa2309e8594b5a52; no voters:
I20260806 12:56:55.793668 30465 leader_election.cc:288] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:55.793720 30468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:55.793859 30468 raft_consensus.cc:686] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [term 1 LEADER]: Becoming Leader. State: Replica: bea5d6a7b8f54d91aa2309e8594b5a52, State: Running, Role: LEADER
I20260806 12:56:55.793968 30465 sys_catalog.cc:564] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:55.793953 30468 consensus_queue.cc:237] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5d6a7b8f54d91aa2309e8594b5a52" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45541 } }
I20260806 12:56:55.794342 30470 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bea5d6a7b8f54d91aa2309e8594b5a52. Latest consensus state: current_term: 1 leader_uuid: "bea5d6a7b8f54d91aa2309e8594b5a52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5d6a7b8f54d91aa2309e8594b5a52" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45541 } } }
I20260806 12:56:55.794430 30470 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:55.794658 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 30398
I20260806 12:56:55.794725 30474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:55.794768 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:55.794970 24908 external_mini_cluster.cc:1428] Setting key 581b1c0a4f9ad86fc9b0fbcaa625ebd7
I20260806 12:56:55.795214 30469 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bea5d6a7b8f54d91aa2309e8594b5a52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5d6a7b8f54d91aa2309e8594b5a52" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45541 } } }
I20260806 12:56:55.795295 30469 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:55.795599 30474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:55.797174 30474 catalog_manager.cc:1347] Generated new cluster ID: e6609346997d4d5b886a1b02e0c79e5e
I20260806 12:56:55.797226 30474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:56:55.801383 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.795481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35962 (local address 127.24.83.62:45541)
0806 12:56:55.795691 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.795700 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.795735 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:56:55.795855 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.795863 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.795965 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.796123 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:55.796718 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.798069 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.799156 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.799800 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.799953 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:55.800588 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:55.800592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:55.800592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:55.800805 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:55.800859 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:55.800873 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:55.800943 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:55.801008 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 12:56:55.801173 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260806 12:56:55.802181 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:56:55.818863 30474 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:55.819459 30474 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:55.828675 30474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bea5d6a7b8f54d91aa2309e8594b5a52: Generated new TSK 0
I20260806 12:56:55.828842 30474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:56:55.839840 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:55.836218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35966 (local address 127.24.83.62:45541)
0806 12:56:55.836259 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:55.836261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:55.836370 (+ 109us) server_negotiation.cc:407] Connection header received
0806 12:56:55.836423 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:55.836423 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:55.836438 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:55.836521 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0806 12:56:55.836621 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.837404 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.838183 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.838501 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.838572 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:55.838712 (+ 140us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 12:56:55.839630 (+ 918us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43315/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 12:56:55.839759 (+ 129us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35966: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43315/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 12:56:55.839917 30486 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35966: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43315/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 12:56:55.841758 30494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:56:55.836144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45541 (local address 127.0.0.1:35966)
0806 12:56:55.836290 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:55.836300 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:55.836359 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:55.836519 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:55.836519 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:55.836521 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 12:56:55.836564 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:55.836565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.837522 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.837522 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:55.838102 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:55.838104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:55.838578 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:55.838578 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:55.838617 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:55.838621 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 12:56:55.839727 (+ 1106us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43315/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 12:56:55.841657 (+ 1930us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.83.62:45541: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43315/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":1763,"thread_start_us":82,"threads_started":1}
I20260806 12:56:55.841971 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 30398
E20260806 12:56:55.851322 30495 webserver.cc:533] Webserver: sslize failed:
E20260806 12:56:55.851377 30495 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T12:56:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (218 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T12:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:55Z Disabled control of system clock
I20260806 12:56:55.888247 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40911
--webserver_interface=127.24.83.62
--webserver_port=0
--builtin_ntp_servers=127.24.83.20:40977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:40911
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/testchainca.cert with env {}
W20260806 12:56:55.966986 30499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:55.967172 30499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:55.967192 30499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:55.968326 30499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:55.968357 30499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:55.968369 30499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:55.968382 30499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:55.968402 30499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 12:56:55.968415 30499 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 12:56:55.968425 30499 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 12:56:55.969801 30499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:40911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:55.969985 30499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:55.970153 30499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:55.974766 30504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:55.974781 30505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:55.974781 30507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:55.975109 30499 server_base.cc:1034] running on GCE node
I20260806 12:56:55.975373 30499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:55.975649 30499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:55.976809 30499 hybrid_clock.cc:648] HybridClock initialized: now 1786021015976788 us; error 31 us; skew 500 ppm
I20260806 12:56:55.977968 30499 webserver.cc:460] Webserver started at http://127.24.83.62:42513/ using document root <none> and password file <none>
I20260806 12:56:55.978215 30499 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:55.978282 30499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:55.978442 30499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:55.979439 30499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "9ca0a05f23e34537925c00d7fae36251"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "021a291035549520d3798ff682fc3884"
server_key_iv: "40d95f825077dc1b666d6d13551adac5"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.979801 30499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "9ca0a05f23e34537925c00d7fae36251"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "021a291035549520d3798ff682fc3884"
server_key_iv: "40d95f825077dc1b666d6d13551adac5"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.981063 30499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:56:55.981778 30513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:55.982010 30499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:56:55.982117 30499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "9ca0a05f23e34537925c00d7fae36251"
format_stamp: "Formatted at 2026-08-06 12:56:55 on dist-test-slave-zkpd"
server_key: "021a291035549520d3798ff682fc3884"
server_key_iv: "40d95f825077dc1b666d6d13551adac5"
server_key_version: "encryptionkey@0"
I20260806 12:56:55.982189 30499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:55.997561 30499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:55.997879 30499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:55.998047 30499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:56.002050 30499 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:40911
I20260806 12:56:56.002100 30570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:40911 every 8 connection(s)
I20260806 12:56:56.002410 30499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:56.003350 30571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:56.005611 30571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251: Bootstrap starting.
I20260806 12:56:56.006116 30571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:56.006371 30571 log.cc:824] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:56.006986 30571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251: No bootstrap required, opened a new log
I20260806 12:56:56.008622 30571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca0a05f23e34537925c00d7fae36251" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40911 } }
I20260806 12:56:56.008762 30571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:56.008785 30571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ca0a05f23e34537925c00d7fae36251, State: Initialized, Role: FOLLOWER
I20260806 12:56:56.008885 30571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca0a05f23e34537925c00d7fae36251" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40911 } }
I20260806 12:56:56.008935 30571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:56.008956 30571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:56.008983 30571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:56.009465 30571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca0a05f23e34537925c00d7fae36251" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40911 } }
I20260806 12:56:56.009552 30571 leader_election.cc:302] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ca0a05f23e34537925c00d7fae36251; no voters:
I20260806 12:56:56.009672 30571 leader_election.cc:288] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:56.009714 30574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:56.009887 30574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [term 1 LEADER]: Becoming Leader. State: Replica: 9ca0a05f23e34537925c00d7fae36251, State: Running, Role: LEADER
I20260806 12:56:56.009964 30571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:56.009989 30574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca0a05f23e34537925c00d7fae36251" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40911 } }
I20260806 12:56:56.010381 30575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ca0a05f23e34537925c00d7fae36251" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca0a05f23e34537925c00d7fae36251" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40911 } } }
I20260806 12:56:56.010537 30575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:56.010517 30576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ca0a05f23e34537925c00d7fae36251. Latest consensus state: current_term: 1 leader_uuid: "9ca0a05f23e34537925c00d7fae36251" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca0a05f23e34537925c00d7fae36251" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40911 } } }
I20260806 12:56:56.010654 30576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:56.011049 30584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:56.011585 30584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:56.012482 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 30499
I20260806 12:56:56.012611 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:56.012850 24908 external_mini_cluster.cc:1428] Setting key 2830033a1f7ebf0af953a5dca8d612ae
I20260806 12:56:56.013240 30584 catalog_manager.cc:1347] Generated new cluster ID: 76012d17699a4c998023804e5d7d227a
I20260806 12:56:56.013294 30584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:56:56.019265 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:56.013519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38798 (local address 127.24.83.62:40911)
0806 12:56:56.013754 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:56.013763 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:56.013794 (+ 31us) server_negotiation.cc:407] Connection header received
0806 12:56:56.013916 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:56.013923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:56.014001 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:56.014142 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:56.014370 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:56.015788 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:56.016873 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:56.017522 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:56.017650 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:56.018323 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:56.018327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:56.018328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:56:56.018537 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:56.018580 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:56:56.018595 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:56.018668 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:56.018737 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 12:56:56.018932 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":80,"threads_started":1}
I20260806 12:56:56.020212 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:56:56.028532 30584 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:56.029184 30584 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:56.036784 30584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ca0a05f23e34537925c00d7fae36251: Generated new TSK 0
I20260806 12:56:56.036974 30584 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:56:56.093518 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:56.044277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38802 (local address 127.24.83.62:40911)
0806 12:56:56.044334 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:56.044336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:56.044410 (+ 74us) server_negotiation.cc:407] Connection header received
0806 12:56:56.044471 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:56.044472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:56.044484 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:56.044556 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0806 12:56:56.044682 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:56.045466 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:56.046148 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:56.046449 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:56.046512 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:56.046674 (+ 162us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 12:56:56.093221 (+ 46547us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 12:56:56.093428 (+ 207us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38802: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 12:56:56.093597 30596 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38802: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 12:56:56.093621 30602 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:56:56.044200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40911 (local address 127.0.0.1:38802)
0806 12:56:56.044357 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:56.044367 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:56.044420 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:56.044560 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:56.044560 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:56.044562 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 12:56:56.044621 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:56.044622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:56.045533 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:56.045533 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:56.046071 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:56.046072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:56.046529 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:56.046529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:56.046574 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:56.046578 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 12:56:56.093376 (+ 46798us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 12:56:56.093500 (+ 124us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.83.62:40911: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":147,"threads_started":1}
I20260806 12:56:56.094028 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 30499
2026-08-06T12:56:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (231 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:56:56 dist-test-slave-zkpd krb5kdc[30614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:56:56 dist-test-slave-zkpd krb5kdc[30614](info): set up 2 sockets
Aug 06 12:56:56 dist-test-slave-zkpd krb5kdc[30614](info): commencing operation
krb5kdc: starting...
W20260806 12:56:57.320829 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:56:57 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:56:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:56:57.436442 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:41357
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:46309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:41357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:57.519701 30627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:57.519945 30627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:57.519968 30627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:57.521147 30627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:57.521188 30627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:57.521202 30627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:57.521215 30627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:57.521270 30627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:57.522667 30627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:46309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:41357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:41357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:57.522872 30627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:57.523103 30627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:57.527810 30635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:57.527783 30633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:57.527804 30632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:57.528048 30627 server_base.cc:1034] running on GCE node
I20260806 12:56:57.528267 30627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:57.528501 30627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:57.529639 30627 hybrid_clock.cc:648] HybridClock initialized: now 1786021017529625 us; error 27 us; skew 500 ppm
Aug 06 12:56:57 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021017, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:57.532029 30627 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:56:57.532486 30627 webserver.cc:460] Webserver started at http://127.24.83.62:36889/ using document root <none> and password file <none>
I20260806 12:56:57.532737 30627 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:57.532805 30627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:57.532963 30627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:57.533978 30627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "c0bdafa6aa8142b9ad817ff8d96216de"
format_stamp: "Formatted at 2026-08-06 12:56:57 on dist-test-slave-zkpd"
server_key: "ce5006c0bcedb8538a24cffe5b7f9aa5"
server_key_iv: "a4c90b67d8d16574694943d6b91dce33"
server_key_version: "encryptionkey@0"
I20260806 12:56:57.534350 30627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "c0bdafa6aa8142b9ad817ff8d96216de"
format_stamp: "Formatted at 2026-08-06 12:56:57 on dist-test-slave-zkpd"
server_key: "ce5006c0bcedb8538a24cffe5b7f9aa5"
server_key_iv: "a4c90b67d8d16574694943d6b91dce33"
server_key_version: "encryptionkey@0"
I20260806 12:56:57.535648 30627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:57.536433 30643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:57.536602 30627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:57.536710 30627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "c0bdafa6aa8142b9ad817ff8d96216de"
format_stamp: "Formatted at 2026-08-06 12:56:57 on dist-test-slave-zkpd"
server_key: "ce5006c0bcedb8538a24cffe5b7f9aa5"
server_key_iv: "a4c90b67d8d16574694943d6b91dce33"
server_key_version: "encryptionkey@0"
I20260806 12:56:57.536784 30627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:57.545850 30627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:57.547540 30627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:57.547696 30627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:57.551972 30627 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:41357
I20260806 12:56:57.551975 30697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:41357 every 8 connection(s)
I20260806 12:56:57.552356 30627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:57.553165 30698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:57.555286 30698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de: Bootstrap starting.
I20260806 12:56:57.555789 30698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:57.556124 30698 log.cc:824] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:57.556695 30698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de: No bootstrap required, opened a new log
I20260806 12:56:57.558182 30698 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bdafa6aa8142b9ad817ff8d96216de" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41357 } }
I20260806 12:56:57.558312 30698 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:57.558333 30698 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0bdafa6aa8142b9ad817ff8d96216de, State: Initialized, Role: FOLLOWER
I20260806 12:56:57.558459 30698 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bdafa6aa8142b9ad817ff8d96216de" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41357 } }
I20260806 12:56:57.558538 30698 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:57.558570 30698 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:57.558600 30698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:57.559056 30698 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bdafa6aa8142b9ad817ff8d96216de" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41357 } }
I20260806 12:56:57.559139 30698 leader_election.cc:302] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0bdafa6aa8142b9ad817ff8d96216de; no voters:
I20260806 12:56:57.559254 30698 leader_election.cc:288] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:57.559333 30701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:57.559501 30698 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:57.559669 30701 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [term 1 LEADER]: Becoming Leader. State: Replica: c0bdafa6aa8142b9ad817ff8d96216de, State: Running, Role: LEADER
I20260806 12:56:57.559756 30701 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bdafa6aa8142b9ad817ff8d96216de" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41357 } }
I20260806 12:56:57.560164 30703 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0bdafa6aa8142b9ad817ff8d96216de. Latest consensus state: current_term: 1 leader_uuid: "c0bdafa6aa8142b9ad817ff8d96216de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bdafa6aa8142b9ad817ff8d96216de" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41357 } } }
I20260806 12:56:57.560181 30702 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0bdafa6aa8142b9ad817ff8d96216de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bdafa6aa8142b9ad817ff8d96216de" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 41357 } } }
I20260806 12:56:57.560263 30703 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:57.560289 30702 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:57.560470 30710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:57.560701 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 30627
I20260806 12:56:57.560818 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:57.561012 24908 external_mini_cluster.cc:1428] Setting key e47a2cea96c79279a00ee5d47155b08f
I20260806 12:56:57.561028 30710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:57.564277 30710 catalog_manager.cc:1347] Generated new cluster ID: 6797ee92ce8e407686929e14731d0fd5
I20260806 12:56:57.564348 30710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:56:57 dist-test-slave-zkpd krb5kdc[30614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:57.572744 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:57.561403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39876 (local address 127.24.83.62:41357)
0806 12:56:57.561585 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:57.561595 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:57.561682 (+ 87us) server_negotiation.cc:407] Connection header received
0806 12:56:57.561845 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:57.561852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:57.561912 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:57.562054 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:57.562815 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:57.563696 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:57.564754 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:57.565148 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:57.565228 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:57.567858 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:57.567863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:57.567863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:57.567874 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:57.570563 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:57.571072 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:57.571075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:57.571075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:57.571229 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:57.571485 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:57.571485 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:57.571486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:57.571864 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:57.572081 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:57.572388 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 12:56:57.572533 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260806 12:56:57.575191 30710 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:57.575847 30710 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:57.581561 30710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0bdafa6aa8142b9ad817ff8d96216de: Generated new TSK 0
I20260806 12:56:57.581717 30710 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:56:57.606442 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41357
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:46309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:57.686950 30725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:57.687181 30725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:57.687222 30725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:57.688448 30725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:57.688508 30725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:57.688551 30725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:56:57.690065 30725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:46309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:57.690343 30725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:57.690595 30725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:57.695479 30734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:57.695423 30732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:57.695421 30731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:57.695677 30725 server_base.cc:1034] running on GCE node
I20260806 12:56:57.695881 30725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:57.696197 30725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:57.697355 30725 hybrid_clock.cc:648] HybridClock initialized: now 1786021017697403 us; error 86 us; skew 500 ppm
Aug 06 12:56:57 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021017, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:57.699748 30725 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:56:57.700212 30725 webserver.cc:460] Webserver started at http://127.24.83.1:44925/ using document root <none> and password file <none>
I20260806 12:56:57.700453 30725 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:57.700522 30725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:57.700670 30725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:57.701689 30725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "0e13c3c5c676414b990ecd376c201b80"
format_stamp: "Formatted at 2026-08-06 12:56:57 on dist-test-slave-zkpd"
server_key: "2d1398dfe3e386d40c46a4b213359a4c"
server_key_iv: "4911805728cd67a708775ce837348762"
server_key_version: "encryptionkey@0"
I20260806 12:56:57.702044 30725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "0e13c3c5c676414b990ecd376c201b80"
format_stamp: "Formatted at 2026-08-06 12:56:57 on dist-test-slave-zkpd"
server_key: "2d1398dfe3e386d40c46a4b213359a4c"
server_key_iv: "4911805728cd67a708775ce837348762"
server_key_version: "encryptionkey@0"
I20260806 12:56:57.703307 30725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 12:56:57.704022 30742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:57.704191 30725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:57.704288 30725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "0e13c3c5c676414b990ecd376c201b80"
format_stamp: "Formatted at 2026-08-06 12:56:57 on dist-test-slave-zkpd"
server_key: "2d1398dfe3e386d40c46a4b213359a4c"
server_key_iv: "4911805728cd67a708775ce837348762"
server_key_version: "encryptionkey@0"
I20260806 12:56:57.704370 30725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:57.716696 30725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:57.718415 30725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:57.718570 30725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:57.718796 30725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:57.719084 30725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:57.719141 30725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:57.719172 30725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:57.719197 30725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:57.724963 30725 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:46593
I20260806 12:56:57.725003 30856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:46593 every 8 connection(s)
I20260806 12:56:57.725306 30725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
Aug 06 12:56:57 dist-test-slave-zkpd krb5kdc[30614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021017, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:57.731043 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 30725
I20260806 12:56:57.731186 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:56:57.731384 24908 external_mini_cluster.cc:1428] Setting key 0739b2f5c9c9acfe266c8e98391fb066
I20260806 12:56:57.734474 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:57.726022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:44421 (local address 127.24.83.62:41357)
0806 12:56:57.726093 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:57.726095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:57.726299 (+ 204us) server_negotiation.cc:407] Connection header received
0806 12:56:57.727152 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:57.727153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:57.727164 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:57.727214 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:57.727753 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:57.728301 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:57.729216 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:57.729420 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:57.729469 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:57.731462 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:57.731464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:57.731465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:57.731474 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:57.733087 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:57.733499 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:57.733500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:57.733500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:57.733556 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:57.733833 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:57.733834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:57.733835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:57.733950 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:57.734027 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:57.734290 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 12:56:57.734376 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:56:57.734632 30859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:57.726003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:41357 (local address 127.24.83.1:44421)
0806 12:56:57.726170 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:57.726208 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:57.727065 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:57.727272 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:57.727275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:57.727508 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:57.727680 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:57.727684 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:57.728377 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:57.728378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:57.729141 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:57.729142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:57.729493 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:57.729494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:57.729604 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:57.730010 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:57.730018 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:57.731367 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:57.733183 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:57.733190 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:57.733192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:57.733410 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:57.733631 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:57.733632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:57.733633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:57.733750 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:57.734019 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:57.734026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:57.734160 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 12:56:57.734268 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 12:56:57.734414 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260806 12:56:57.735090 30857 heartbeater.cc:347] Connected to a master server at 127.24.83.62:41357
I20260806 12:56:57.735163 30857 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:57.735345 30857 heartbeater.cc:511] Master 127.24.83.62:41357 requested a full tablet report, sending...
I20260806 12:56:57.735985 30662 ts_manager.cc:194] Registered new tserver with Master: 0e13c3c5c676414b990ecd376c201b80 (127.24.83.1:46593)
I20260806 12:56:57.736994 30662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:44421
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:56:57.756270 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41357
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:46309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:57.836198 30862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:57.836396 30862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:57.836417 30862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:57.837471 30862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:57.837502 30862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:57.837527 30862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:56:57.838874 30862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:46309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:57.839108 30862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:57.839322 30862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:57.844275 30872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:57.844326 30869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:57.844272 30870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:57.844498 30862 server_base.cc:1034] running on GCE node
I20260806 12:56:57.844693 30862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:57.844880 30862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:57.846019 30862 hybrid_clock.cc:648] HybridClock initialized: now 1786021017846006 us; error 27 us; skew 500 ppm
Aug 06 12:56:57 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021017, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:57.848491 30862 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:56:57.849009 30862 webserver.cc:460] Webserver started at http://127.24.83.2:37057/ using document root <none> and password file <none>
I20260806 12:56:57.849236 30862 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:57.849306 30862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:57.849455 30862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:57.850517 30862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "769a3f3cee8e4097b82f9cb36b56d576"
format_stamp: "Formatted at 2026-08-06 12:56:57 on dist-test-slave-zkpd"
server_key: "e4d56904a7ccffe3e9d0f9b443a5eaec"
server_key_iv: "a2a6fdf6257288406016ee5b7ebcf23d"
server_key_version: "encryptionkey@0"
I20260806 12:56:57.850883 30862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "769a3f3cee8e4097b82f9cb36b56d576"
format_stamp: "Formatted at 2026-08-06 12:56:57 on dist-test-slave-zkpd"
server_key: "e4d56904a7ccffe3e9d0f9b443a5eaec"
server_key_iv: "a2a6fdf6257288406016ee5b7ebcf23d"
server_key_version: "encryptionkey@0"
I20260806 12:56:57.852414 30862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 12:56:57.853161 30879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:57.853327 30862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:57.853422 30862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "769a3f3cee8e4097b82f9cb36b56d576"
format_stamp: "Formatted at 2026-08-06 12:56:57 on dist-test-slave-zkpd"
server_key: "e4d56904a7ccffe3e9d0f9b443a5eaec"
server_key_iv: "a2a6fdf6257288406016ee5b7ebcf23d"
server_key_version: "encryptionkey@0"
I20260806 12:56:57.853526 30862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:57.875352 30862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:57.877736 30862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:57.877921 30862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:57.878140 30862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:57.878428 30862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:57.878460 30862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:57.878512 30862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:57.878547 30862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:57.884338 30862 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:44981
I20260806 12:56:57.884395 30993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:44981 every 8 connection(s)
I20260806 12:56:57.884689 30862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:56:57 dist-test-slave-zkpd krb5kdc[30614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021017, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:57.890699 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 30862
I20260806 12:56:57.890801 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:56:57.890980 24908 external_mini_cluster.cc:1428] Setting key ceff432e8de6d5c9c3fad39e698fc0c6
I20260806 12:56:57.893889 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:57.885413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:45039 (local address 127.24.83.62:41357)
0806 12:56:57.885511 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:57.885514 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:57.885682 (+ 168us) server_negotiation.cc:407] Connection header received
0806 12:56:57.886590 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:57.886592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:57.886606 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:57.886663 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:57.887249 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:57.887771 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:57.888801 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:57.889014 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:57.889100 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:57.891021 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:57.891022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:57.891023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:57.891032 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:57.892479 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:57.892871 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:57.892872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:57.892872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:57.892928 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:57.893226 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:57.893227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:57.893227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:57.893348 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:57.893410 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:57.893742 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 12:56:57.893828 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:56:57.894137 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:57.885384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:41357 (local address 127.24.83.2:45039)
0806 12:56:57.885556 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:57.885599 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:57.886481 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:57.886731 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:57.886734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:57.886971 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:57.887152 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:57.887155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:57.887878 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:57.887879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:57.888709 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:57.888711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:57.889110 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:57.889111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:57.889228 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:57.889645 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:57.889652 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:57.890935 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:57.892576 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:57.892582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:57.892584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:57.892792 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:57.893004 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:57.893004 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:57.893005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:57.893144 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:57.893419 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:57.893425 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:57.893581 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 12:56:57.893704 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 12:56:57.893906 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260806 12:56:57.894636 30994 heartbeater.cc:347] Connected to a master server at 127.24.83.62:41357
I20260806 12:56:57.894721 30994 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:57.894904 30994 heartbeater.cc:511] Master 127.24.83.62:41357 requested a full tablet report, sending...
I20260806 12:56:57.895295 30662 ts_manager.cc:194] Registered new tserver with Master: 769a3f3cee8e4097b82f9cb36b56d576 (127.24.83.2:44981)
I20260806 12:56:57.895664 30662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:45039
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:56:57.914361 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41357
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:46309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:58.006492 30999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:58.006706 30999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:58.006744 30999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:58.008005 30999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:58.008060 30999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:58.008104 30999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:56:58.009574 30999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:46309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:58.009826 30999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:58.010059 30999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:58.014854 31004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:58.014901 31009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:58.015007 30999 server_base.cc:1034] running on GCE node
W20260806 12:56:58.015224 31005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:58.015549 30999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:58.015776 30999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:58.016916 30999 hybrid_clock.cc:648] HybridClock initialized: now 1786021018016926 us; error 50 us; skew 500 ppm
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:58.019371 30999 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:56:58.019825 30999 webserver.cc:460] Webserver started at http://127.24.83.3:33001/ using document root <none> and password file <none>
I20260806 12:56:58.020061 30999 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:58.020128 30999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:58.020277 30999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:58.021260 30999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "163ef0d1746b48e297a230a1f79606bc"
format_stamp: "Formatted at 2026-08-06 12:56:58 on dist-test-slave-zkpd"
server_key: "18ed1d7ee9c5bfeb36ab50c6f9133871"
server_key_iv: "cd81bcd11ccf4835c8642c5cfe1b6f36"
server_key_version: "encryptionkey@0"
I20260806 12:56:58.021623 30999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "163ef0d1746b48e297a230a1f79606bc"
format_stamp: "Formatted at 2026-08-06 12:56:58 on dist-test-slave-zkpd"
server_key: "18ed1d7ee9c5bfeb36ab50c6f9133871"
server_key_iv: "cd81bcd11ccf4835c8642c5cfe1b6f36"
server_key_version: "encryptionkey@0"
I20260806 12:56:58.022938 30999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:58.023591 31016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:58.023756 30999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:58.023849 30999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "163ef0d1746b48e297a230a1f79606bc"
format_stamp: "Formatted at 2026-08-06 12:56:58 on dist-test-slave-zkpd"
server_key: "18ed1d7ee9c5bfeb36ab50c6f9133871"
server_key_iv: "cd81bcd11ccf4835c8642c5cfe1b6f36"
server_key_version: "encryptionkey@0"
I20260806 12:56:58.023960 30999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:58.036582 30999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:58.038856 30999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:58.039018 30999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:58.039263 30999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:58.039582 30999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:58.039638 30999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:58.039696 30999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:58.039723 30999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:58.045439 30999 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:34953
I20260806 12:56:58.045482 31130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:34953 every 8 connection(s)
I20260806 12:56:58.045785 30999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:56:58.048801 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 30999
I20260806 12:56:58.048905 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:56:58.049096 24908 external_mini_cluster.cc:1428] Setting key 32c73754c3ef95c11c817aecd339125b
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:56:58.055541 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:58.046470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:43379 (local address 127.24.83.62:41357)
0806 12:56:58.046551 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:58.046554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:58.046794 (+ 240us) server_negotiation.cc:407] Connection header received
0806 12:56:58.047749 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:58.047751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:58.047766 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:58.047825 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:58.048494 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:58.049319 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:58.050604 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:58.050828 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:58.050906 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:58.052940 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:58.052942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:58.052942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:58.052949 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:58.054391 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:58.054772 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:58.054773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:58.054774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:58.054831 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:58.055066 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:58.055066 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:58.055066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:58.055162 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:58.055211 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:58.055418 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 12:56:58.055482 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:56:58.055742 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:58.046417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:41357 (local address 127.24.83.3:43379)
0806 12:56:58.046642 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:58.046704 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:58.047625 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:58.047936 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:58.047940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:58.048206 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:58.048395 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:58.048400 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:58.049427 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:58.049429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:58.050512 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:58.050516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:58.050914 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:58.050915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:58.051043 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:58.051420 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:58.051427 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:58.052856 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:58.054463 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:58.054468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:58.054470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:58.054675 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:58.054898 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:58.054899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:58.054899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:58.054993 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:58.055222 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:58.055226 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:58.055327 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 12:56:58.055403 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 12:56:58.055510 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":107,"threads_started":1}
I20260806 12:56:58.056267 31131 heartbeater.cc:347] Connected to a master server at 127.24.83.62:41357
I20260806 12:56:58.056386 31131 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:58.056578 31131 heartbeater.cc:511] Master 127.24.83.62:41357 requested a full tablet report, sending...
I20260806 12:56:58.056931 30662 ts_manager.cc:194] Registered new tserver with Master: 163ef0d1746b48e297a230a1f79606bc (127.24.83.3:34953)
I20260806 12:56:58.057323 30662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:43379
I20260806 12:56:58.060915 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/test-admin.keytab.
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:58.076188 24908 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 12:56:58.077694 31138 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:56:58.076917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36534 (local address 127.24.83.1:46593)
0806 12:56:58.077091 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:58.077096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:58.077104 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:56:58.077127 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:58.077129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:58.077141 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:58.077226 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:58.077615 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36534: BlockingRecv error: recv got EOF from 127.0.0.1:36534 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
W20260806 12:56:58.077760 24908 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5cc' of type FILE
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:58.078691 24908 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:58.079494 24908 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:56:58.087253 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:58.080009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.24.83.1:46593)
0806 12:56:58.080062 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:58.080064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:58.080074 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:56:58.080122 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:58.080123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:58.080133 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:58.080205 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:58.080597 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:58.081347 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:58.082278 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:58.082586 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:58.082665 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:58.084698 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:58.084706 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:58.084707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:58.084713 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:58.086094 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:58.086456 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:58.086459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:58.086459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:58.086515 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:58.086714 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:58.086715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:58.086715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:58.086845 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:58.087028 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:58.087074 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 12:56:58.087146 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 12:56:58.088855 31138 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:56:58.088392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.24.83.1:46593)
0806 12:56:58.088424 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:58.088425 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:58.088449 (+ 24us) server_negotiation.cc:407] Connection header received
0806 12:56:58.088497 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:58.088498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:58.088506 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:58.088551 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:58.088805 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36548: BlockingRecv error: recv got EOF from 127.0.0.1:36548 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 12:56:58.088953 24908 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5cc' of type FILE
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:58.089794 24908 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:58.090574 24908 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:56:58.097062 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:58.091051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36558 (local address 127.24.83.1:46593)
0806 12:56:58.091114 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:58.091115 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:58.091153 (+ 38us) server_negotiation.cc:407] Connection header received
0806 12:56:58.091209 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:58.091210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:58.091216 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:58.091281 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:58.091624 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:58.092169 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:58.092809 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:58.093004 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:58.093051 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:58.094984 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:58.094985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:58.094986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:58.094991 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:58.096152 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:58.096444 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:58.096445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:58.096445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:58.096489 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:58.096642 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:58.096642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:58.096643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:58.096716 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:58.096778 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:58.096872 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 12:56:58.096931 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 12:56:58.098388 31138 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:56:58.097889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36572 (local address 127.24.83.1:46593)
0806 12:56:58.097959 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:58.097960 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:58.098002 (+ 42us) server_negotiation.cc:407] Connection header received
0806 12:56:58.098050 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:58.098050 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:58.098057 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:58.098099 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:58.098345 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36572: BlockingRecv error: recv got EOF from 127.0.0.1:36572 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 12:56:58.098536 24908 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786020991335182-24908-0/krb5kdc/krb5cc)
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:58.099484 24908 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:58.100411 24908 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[30614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:56:58.107095 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:58.100965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.24.83.1:46593)
0806 12:56:58.101012 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:58.101014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:58.101025 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:56:58.101094 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:58.101094 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:58.101101 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:58.101139 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:58.101483 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:58.101905 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:58.102829 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:58.103006 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:58.103052 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:58.105010 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:58.105011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:58.105012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:58.105019 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:58.106117 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:58.106413 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:58.106414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:58.106414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:58.106460 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:58.106659 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:58.106659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:58.106660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:58.106739 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:58.106803 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:58.106938 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 12:56:58.107007 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 12:56:58.107734 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 30725
I20260806 12:56:58.113183 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 30862
I20260806 12:56:58.119292 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 30999
I20260806 12:56:58.124351 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 30627
2026-08-06T12:56:58Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1855 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[31153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[31153](info): set up 2 sockets
Aug 06 12:56:58 dist-test-slave-zkpd krb5kdc[31153](info): commencing operation
krb5kdc: starting...
W20260806 12:56:59.169242 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:56:59 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:56:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:56:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.62.keytab.
Entry for principal oryx/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.62.keytab.
I20260806 12:56:59.284911 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:35127
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.62.keytab
--principal=oryx/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:35127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:59.364676 31167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:59.364866 31167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:59.364935 31167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:59.366076 31167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:56:59.366114 31167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:59.366128 31167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:59.366174 31167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:56:59.366192 31167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:56:59.367602 31167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:35127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.62.keytab
--principal=oryx/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:35127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:59.367844 31167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:59.368095 31167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:59.372663 31172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:59.372742 31174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:59.372761 31167 server_base.cc:1034] running on GCE node
W20260806 12:56:59.372663 31176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:59.373142 31167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:59.373423 31167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:59.374562 31167 hybrid_clock.cc:648] HybridClock initialized: now 1786021019374546 us; error 35 us; skew 500 ppm
Aug 06 12:56:59 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:59.377192 31167 init.cc:377] Logged in from keytab as oryx/127.24.83.62@KRBTEST.COM (short username oryx)
I20260806 12:56:59.377637 31167 webserver.cc:460] Webserver started at http://127.24.83.62:34013/ using document root <none> and password file <none>
I20260806 12:56:59.377843 31167 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:59.377887 31167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:59.378048 31167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:59.379067 31167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "70b5b5dd010144199787f551a70ea8a6"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "2b820c8458798e6a947e70940c755562"
server_key_iv: "acc6ad4c3272b0504f3f9995e27272ac"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.379451 31167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "70b5b5dd010144199787f551a70ea8a6"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "2b820c8458798e6a947e70940c755562"
server_key_iv: "acc6ad4c3272b0504f3f9995e27272ac"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.380805 31167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:56:59.381513 31183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:59.381669 31167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:59.381768 31167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "70b5b5dd010144199787f551a70ea8a6"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "2b820c8458798e6a947e70940c755562"
server_key_iv: "acc6ad4c3272b0504f3f9995e27272ac"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.381836 31167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:59.392186 31167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:59.393896 31167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:59.394045 31167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:59.398138 31167 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:35127
I20260806 12:56:59.398170 31235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:35127 every 8 connection(s)
I20260806 12:56:59.398542 31167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:56:59.399051 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 31167
I20260806 12:56:59.399201 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:56:59.399459 24908 external_mini_cluster.cc:1428] Setting key 01a826ae7253a440be545abe265f7f48
I20260806 12:56:59.399721 31236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:56:59.402421 31236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6: Bootstrap starting.
I20260806 12:56:59.403026 31236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6: Neither blocks nor log segments found. Creating new log.
I20260806 12:56:59.403301 31236 log.cc:824] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6: Log is configured to *not* fsync() on all Append() calls
I20260806 12:56:59.404075 31236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6: No bootstrap required, opened a new log
Aug 06 12:56:59 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:56:59.406530 31236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } }
I20260806 12:56:59.406713 31236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:56:59.406757 31236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70b5b5dd010144199787f551a70ea8a6, State: Initialized, Role: FOLLOWER
I20260806 12:56:59.406913 31236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } }
I20260806 12:56:59.407050 31236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:56:59.407089 31236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:56:59.407145 31236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:56:59.407846 31236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } }
I20260806 12:56:59.408012 31236 leader_election.cc:302] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70b5b5dd010144199787f551a70ea8a6; no voters:
I20260806 12:56:59.408165 31236 leader_election.cc:288] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:56:59.408210 31241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:56:59.408412 31241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 1 LEADER]: Becoming Leader. State: Replica: 70b5b5dd010144199787f551a70ea8a6, State: Running, Role: LEADER
I20260806 12:56:59.408445 31236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:56:59.408528 31241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } }
I20260806 12:56:59.408852 31241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } }
I20260806 12:56:59.408944 31241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:59.409107 31241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70b5b5dd010144199787f551a70ea8a6. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } }
I20260806 12:56:59.409188 31241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:56:59.409313 31249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:56:59.409847 31249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:56:59.410956 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:59.399928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.24.83.62:35127)
0806 12:56:59.400112 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:59.400121 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:59.400151 (+ 30us) server_negotiation.cc:407] Connection header received
0806 12:56:59.400781 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:59.400788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:59.400831 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:59.400971 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:59.401580 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.402415 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.403284 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.403844 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.404289 (+ 445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:59.406509 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:59.406531 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:59.406532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:59.406544 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:59.408723 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:59.409189 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:59.409192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:59.409192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:59.409312 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:59.409557 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:59.409557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:59.409558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:59.409901 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:59.410554 (+ 653us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:59.410627 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 12:56:59.410754 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260806 12:56:59.411779 31249 catalog_manager.cc:1347] Generated new cluster ID: c298d5eafbc745d6945d505298714ac0
I20260806 12:56:59.411832 31249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:56:59.426307 31249 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:56:59.426857 31249 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:56:59.433300 31249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6: Generated new TSK 0
I20260806 12:56:59.433466 31249 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab.
Entry for principal oryx/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab.
I20260806 12:56:59.466471 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35127
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab
--principal=oryx/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:59.547183 31263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:59.547371 31263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:59.547401 31263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:59.548535 31263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:59.548575 31263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:59.548623 31263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:56:59.550073 31263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab
--principal=oryx/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:59.550319 31263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:59.550551 31263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:56:59.555631 31263 server_base.cc:1034] running on GCE node
W20260806 12:56:59.555599 31268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:59.555577 31269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:59.555599 31271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:59.556097 31263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:59.556331 31263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:59.557462 31263 hybrid_clock.cc:648] HybridClock initialized: now 1786021019557445 us; error 35 us; skew 500 ppm
Aug 06 12:56:59 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:59.559990 31263 init.cc:377] Logged in from keytab as oryx/127.24.83.1@KRBTEST.COM (short username oryx)
I20260806 12:56:59.560406 31263 webserver.cc:460] Webserver started at http://127.24.83.1:43873/ using document root <none> and password file <none>
I20260806 12:56:59.560634 31263 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:59.560695 31263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:59.560839 31263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:59.561859 31263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "a30d865941464d91bd03c5cc1cb2bee2"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "74e5eae5802962e79de7bcca0d71eeb8"
server_key_iv: "5ed3f7406ac32597ab3d2451b5fae959"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.562222 31263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "a30d865941464d91bd03c5cc1cb2bee2"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "74e5eae5802962e79de7bcca0d71eeb8"
server_key_iv: "5ed3f7406ac32597ab3d2451b5fae959"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.563506 31263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 12:56:59.564261 31278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:59.564425 31263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:59.564518 31263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "a30d865941464d91bd03c5cc1cb2bee2"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "74e5eae5802962e79de7bcca0d71eeb8"
server_key_iv: "5ed3f7406ac32597ab3d2451b5fae959"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.564592 31263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:59.573457 31263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:59.575665 31263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:59.575814 31263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:59.576083 31263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:59.576370 31263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:59.576421 31263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:59.576460 31263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:59.576491 31263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:59.582182 31263 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:37175
I20260806 12:56:59.582240 31397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:37175 every 8 connection(s)
I20260806 12:56:59.582525 31263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
Aug 06 12:56:59 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:56:59.590912 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:59.583188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:54693 (local address 127.24.83.62:35127)
0806 12:56:59.583249 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:59.583251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:59.583458 (+ 207us) server_negotiation.cc:407] Connection header received
0806 12:56:59.583958 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:59.583959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:59.583972 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:59.584024 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:59.584974 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.585520 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.586452 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.586634 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.586684 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:59.588684 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:59.588686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:59.588686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:59.588693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:59.589859 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:59.590200 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:59.590200 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:59.590201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:59.590246 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:59.590435 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:59.590435 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:59.590435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:59.590528 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:59.590575 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:59.590776 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 12:56:59.590849 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 12:56:59.591048 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:59.583166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.1:54693)
0806 12:56:59.583344 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:59.583379 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:59.583831 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:59.584073 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:59.584077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:59.584700 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:59.584882 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:59.584895 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.585602 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.585603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:59.586385 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:59.586387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.586701 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.586701 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:59.586795 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:59.587282 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:59.587290 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:59.588605 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:59.589924 (+ 1319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:59.589930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:59.589932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:59.590141 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:59.590296 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:59.590297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:59.590297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:59.590385 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:59.590575 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:59.590580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:59.590681 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 12:56:59.590755 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 12:56:59.590861 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":111,"threads_started":1}
I20260806 12:56:59.591449 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 31263
I20260806 12:56:59.591602 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:56:59.591638 31398 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35127
I20260806 12:56:59.591727 31398 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:59.591848 24908 external_mini_cluster.cc:1428] Setting key 5ecfc0cfaa0348cdb7cd96e0275bc492
I20260806 12:56:59.591946 31398 heartbeater.cc:511] Master 127.24.83.62:35127 requested a full tablet report, sending...
I20260806 12:56:59.592541 31199 ts_manager.cc:194] Registered new tserver with Master: a30d865941464d91bd03c5cc1cb2bee2 (127.24.83.1:37175)
I20260806 12:56:59.593443 31199 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:54693
WARNING: no policy specified for oryx/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab.
Entry for principal oryx/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab.
I20260806 12:56:59.616032 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35127
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab
--principal=oryx/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:59.695592 31403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:59.695819 31403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:59.695858 31403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:59.696982 31403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:59.697044 31403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:59.697090 31403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:56:59.698505 31403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab
--principal=oryx/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:59.698757 31403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:59.698980 31403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:59.703824 31409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:59.703809 31408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:59.703879 31403 server_base.cc:1034] running on GCE node
W20260806 12:56:59.703824 31411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:59.704289 31403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:59.704543 31403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:59.705680 31403 hybrid_clock.cc:648] HybridClock initialized: now 1786021019705662 us; error 33 us; skew 500 ppm
Aug 06 12:56:59 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:59.708153 31403 init.cc:377] Logged in from keytab as oryx/127.24.83.2@KRBTEST.COM (short username oryx)
I20260806 12:56:59.708657 31403 webserver.cc:460] Webserver started at http://127.24.83.2:36979/ using document root <none> and password file <none>
I20260806 12:56:59.708945 31403 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:59.708997 31403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:59.709136 31403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:59.710259 31403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "bdbf455301784493b51959a559bb65d2"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "8135dad94f97574a738f9de893a3d152"
server_key_iv: "9498e96daa3c70d86eabbfb81931ca87"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.710614 31403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "bdbf455301784493b51959a559bb65d2"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "8135dad94f97574a738f9de893a3d152"
server_key_iv: "9498e96daa3c70d86eabbfb81931ca87"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.711925 31403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260806 12:56:59.712646 31418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:59.712791 31403 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:59.712878 31403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "bdbf455301784493b51959a559bb65d2"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "8135dad94f97574a738f9de893a3d152"
server_key_iv: "9498e96daa3c70d86eabbfb81931ca87"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.712936 31403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:59.729924 31403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:59.732182 31403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:59.732290 31403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:59.732462 31403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:59.732695 31403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:59.732724 31403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:59.732745 31403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:59.732757 31403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:59.738179 31403 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:37379
I20260806 12:56:59.738237 31538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:37379 every 8 connection(s)
I20260806 12:56:59.738551 31403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:56:59.740518 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 31403
I20260806 12:56:59.740625 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:56:59.740816 24908 external_mini_cluster.cc:1428] Setting key ab1ff0f365bd7d6059a5b7c2b989fb78
Aug 06 12:56:59 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:56:59.748548 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:59.739261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:33771 (local address 127.24.83.62:35127)
0806 12:56:59.739324 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:59.739327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:59.739520 (+ 193us) server_negotiation.cc:407] Connection header received
0806 12:56:59.740042 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:59.740043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:59.740056 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:59.740114 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:59.741229 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.741805 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.742820 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.743086 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.743147 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:59.745651 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:59.745653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:59.745653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:59.745660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:59.747071 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:59.747559 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:59.747559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:59.747560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:59.747608 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:59.747873 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:59.747874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:59.747874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:59.748024 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:59.748077 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:59.748427 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 12:56:59.748485 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 12:56:59.748864 31541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:59.739220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.2:33771)
0806 12:56:59.739384 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:59.739436 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:59.739941 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:59.740185 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:59.740189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:59.740828 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:59.741112 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:59.741119 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.741902 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.741903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:59.742700 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:59.742703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.743172 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.743173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:59.743327 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:59.743818 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:59.743826 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:59.745550 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:59.747153 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:59.747161 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:59.747163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:59.747480 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:59.747672 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:59.747673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:59.747673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:59.747808 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:59.748091 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:59.748097 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:59.748265 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 12:56:59.748394 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 12:56:59.748561 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260806 12:56:59.749406 31539 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35127
I20260806 12:56:59.749509 31539 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:59.749689 31539 heartbeater.cc:511] Master 127.24.83.62:35127 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.24.83.3@KRBTEST.COM; defaulting to no policy
I20260806 12:56:59.750072 31199 ts_manager.cc:194] Registered new tserver with Master: bdbf455301784493b51959a559bb65d2 (127.24.83.2:37379)
I20260806 12:56:59.750449 31199 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:33771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab.
Entry for principal oryx/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab.
I20260806 12:56:59.764686 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35127
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab
--principal=oryx/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:56:59.845927 31544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:56:59.846146 31544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:56:59.846194 31544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:56:59.847317 31544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:56:59.847379 31544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:56:59.847424 31544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:56:59.848910 31544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab
--principal=oryx/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:56:59.849171 31544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:56:59.849395 31544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:56:59.854166 31549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:56:59.854175 31550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:59.854204 31544 server_base.cc:1034] running on GCE node
W20260806 12:56:59.854187 31552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:56:59.854547 31544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:56:59.854828 31544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:56:59.855949 31544 hybrid_clock.cc:648] HybridClock initialized: now 1786021019855929 us; error 35 us; skew 500 ppm
Aug 06 12:56:59 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:56:59.858517 31544 init.cc:377] Logged in from keytab as oryx/127.24.83.3@KRBTEST.COM (short username oryx)
I20260806 12:56:59.858982 31544 webserver.cc:460] Webserver started at http://127.24.83.3:45203/ using document root <none> and password file <none>
I20260806 12:56:59.859192 31544 fs_manager.cc:359] Metadata directory not provided
I20260806 12:56:59.859252 31544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:56:59.859391 31544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:56:59.860417 31544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "cf29cfa522fab06a091bdfc8ae6564a9"
server_key_iv: "5ab9abe6fcb8e155fed8752bfc85b5b8"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.860761 31544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "cf29cfa522fab06a091bdfc8ae6564a9"
server_key_iv: "5ab9abe6fcb8e155fed8752bfc85b5b8"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.862001 31544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:56:59.862670 31559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:59.862833 31544 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:56:59.862922 31544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "cf29cfa522fab06a091bdfc8ae6564a9"
server_key_iv: "5ab9abe6fcb8e155fed8752bfc85b5b8"
server_key_version: "encryptionkey@0"
I20260806 12:56:59.862995 31544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:56:59.874846 31544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:56:59.876578 31544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:56:59.876724 31544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:56:59.876933 31544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:56:59.877223 31544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:56:59.877274 31544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:59.877307 31544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:56:59.877342 31544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:56:59.883177 31544 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:40855
I20260806 12:56:59.883220 31677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:40855 every 8 connection(s)
I20260806 12:56:59.883514 31544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:56:59.889034 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 31544
Aug 06 12:56:59 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:56:59.889146 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:56:59.889335 24908 external_mini_cluster.cc:1428] Setting key e503e58f08d09a402331f5e2844f4e83
I20260806 12:56:59.892055 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:59.884190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:59803 (local address 127.24.83.62:35127)
0806 12:56:59.884238 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 12:56:59.884240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:56:59.884462 (+ 222us) server_negotiation.cc:407] Connection header received
0806 12:56:59.884890 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:56:59.884891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:56:59.884900 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:56:59.884986 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:56:59.885838 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.886347 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.887298 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.887476 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.887525 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:59.889680 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:56:59.889681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:56:59.889681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:56:59.889690 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:56:59.890968 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:59.891308 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:59.891308 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:59.891309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:59.891355 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:56:59.891550 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:56:59.891550 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:56:59.891551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:56:59.891643 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:56:59.891690 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 12:56:59.891924 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 12:56:59.891996 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:56:59.892274 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:56:59.884170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.3:59803)
0806 12:56:59.884335 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 12:56:59.884374 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:56:59.884811 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:56:59.885048 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:56:59.885051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:56:59.885578 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:56:59.885756 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:59.885760 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.886447 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.886448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:59.887228 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:56:59.887230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:56:59.887541 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:56:59.887542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:56:59.887637 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:56:59.888109 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:56:59.888117 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:56:59.889585 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:56:59.891035 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:59.891040 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:59.891042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:59.891242 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:59.891407 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:56:59.891407 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:56:59.891407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:56:59.891501 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:56:59.891691 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:56:59.891695 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:56:59.891792 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 12:56:59.891865 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 12:56:59.892037 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260806 12:56:59.892695 31678 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35127
I20260806 12:56:59.892802 31678 heartbeater.cc:464] Registering TS with master...
I20260806 12:56:59.892963 31678 heartbeater.cc:511] Master 127.24.83.62:35127 requested a full tablet report, sending...
I20260806 12:56:59.893301 31199 ts_manager.cc:194] Registered new tserver with Master: a45e4d5fb78f4ae4b94b9dbcbde1a08f (127.24.83.3:40855)
I20260806 12:56:59.893653 31199 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:59803
I20260806 12:56:59.900908 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.83.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.61.keytab.
Entry for principal oryx/127.24.83.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.61.keytab.
I20260806 12:56:59.924539 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.61:40135
--webserver_interface=127.24.83.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.61.keytab
--principal=oryx/127.24.83.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:35127,127.24.83.61:40135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:00.005236 31683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:00.005436 31683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:00.005457 31683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:00.006515 31683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:57:00.006547 31683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:00.006561 31683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:00.006573 31683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:57:00.006585 31683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:57:00.008011 31683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:35127,127.24.83.61:40135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.61.keytab
--principal=oryx/127.24.83.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.61:40135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.83.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
W20260806 12:57:00.008107 31683 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.83.62:35127,127.24.83.61:40135'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 12:57:00.008265 31683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:00.008529 31683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:00.013352 31691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:00.013414 31689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:00.013418 31688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:00.013532 31683 server_base.cc:1034] running on GCE node
I20260806 12:57:00.013814 31683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:00.014102 31683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:00.015272 31683 hybrid_clock.cc:648] HybridClock initialized: now 1786021020015263 us; error 36 us; skew 500 ppm
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:00.022457 31683 init.cc:377] Logged in from keytab as oryx/127.24.83.61@KRBTEST.COM (short username oryx)
I20260806 12:57:00.022848 31683 webserver.cc:460] Webserver started at http://127.24.83.61:36283/ using document root <none> and password file <none>
I20260806 12:57:00.023037 31683 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:00.023078 31683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:00.023169 31683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:00.024262 31683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data/instance:
uuid: "7ae452e605104e909ae0c5ca954c82ba"
format_stamp: "Formatted at 2026-08-06 12:57:00 on dist-test-slave-zkpd"
server_key: "b4b4edc19b4e274469f7eaaa91a52e4a"
server_key_iv: "771ee494c6e333cb3728ef22dda31ef9"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.024569 31683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/wal/instance:
uuid: "7ae452e605104e909ae0c5ca954c82ba"
format_stamp: "Formatted at 2026-08-06 12:57:00 on dist-test-slave-zkpd"
server_key: "b4b4edc19b4e274469f7eaaa91a52e4a"
server_key_iv: "771ee494c6e333cb3728ef22dda31ef9"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.025858 31683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:00.026556 31698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.026755 31683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:00.026849 31683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/wal
uuid: "7ae452e605104e909ae0c5ca954c82ba"
format_stamp: "Formatted at 2026-08-06 12:57:00 on dist-test-slave-zkpd"
server_key: "b4b4edc19b4e274469f7eaaa91a52e4a"
server_key_iv: "771ee494c6e333cb3728ef22dda31ef9"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.026906 31683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:00.047147 31683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:00.049460 31683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:00.049582 31683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:00.053298 31683 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.61:40135
I20260806 12:57:00.053345 31751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.61:40135 every 8 connection(s)
I20260806 12:57:00.053658 31683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
I20260806 12:57:00.054616 31752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:00.056149 31752 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:57:00.059028 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 31683
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.61@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.83.61@KRBTEST.COM
I20260806 12:57:00.065973 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.056773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53092 (local address 127.24.83.62:35127)
0806 12:57:00.056859 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.056861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.057121 (+ 260us) server_negotiation.cc:407] Connection header received
0806 12:57:00.057539 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.057540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.057550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.057600 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.058107 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.058586 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.059978 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.060282 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.060365 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.062741 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.062742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.062743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.062750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.064406 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.064900 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.064901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.064901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.064955 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.065225 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.065227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.065227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.065427 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.065522 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.065777 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.065865 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:57:00.066069 31754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.056740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:53092)
0806 12:57:00.057006 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.057040 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.057461 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.057647 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.057650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.057861 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.058039 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.058043 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.058657 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.058657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.059878 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.059881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.060366 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.060366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.060468 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.060920 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.060929 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.062651 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.064481 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.064489 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.064492 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.064822 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.065020 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.065021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.065022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.065160 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.065524 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.065532 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.065650 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.065732 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.065890 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":178,"threads_started":1}
I20260806 12:57:00.067029 31752 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:57:00.069357 31756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.059576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58912 (local address 127.24.83.61:40135)
0806 12:57:00.059860 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.059864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.059882 (+ 18us) server_negotiation.cc:407] Connection header received
0806 12:57:00.059931 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.059933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.059960 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.060027 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.060751 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.061758 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.062810 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.063270 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.063337 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.066366 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.066373 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.066374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.066380 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.068189 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.068565 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.068567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.068567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.068626 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.068805 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.068805 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.068806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.068967 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.069123 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.069162 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.069232 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":82,"threads_started":1}
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.61@KRBTEST.COM for oryx/127.24.83.61@KRBTEST.COM
I20260806 12:57:00.073832 31754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.067283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:40135 (local address 127.0.0.1:58916)
0806 12:57:00.067320 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.067329 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.067382 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.067591 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.067592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.067726 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.067783 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.067795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.068558 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.068559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.069299 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.069301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.069745 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.069746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.069814 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.070223 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.070228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.071270 (+ 1042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.073101 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.073102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.073102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.073254 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.073417 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.073417 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.073417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.073464 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.073622 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.073623 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.073654 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.073712 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.073777 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 12:57:00.074007 31757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.067322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.24.83.61:40135)
0806 12:57:00.067472 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.067474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.067481 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:57:00.067504 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.067504 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.067512 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.067568 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.067910 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.068479 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.069383 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.069682 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.069732 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.071347 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.071348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.071348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.071354 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.073035 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.073315 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.073316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.073316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.073364 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.073501 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.073501 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.073501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.073579 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.073621 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.073727 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.073831 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260806 12:57:00.075212 31752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: Bootstrap starting.
I20260806 12:57:00.076004 31752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:00.076292 31752 log.cc:824] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:00.076949 31752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: No bootstrap required, opened a new log
I20260806 12:57:00.078253 31752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } }
I20260806 12:57:00.078373 31752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:00.078416 31752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae452e605104e909ae0c5ca954c82ba, State: Initialized, Role: FOLLOWER
I20260806 12:57:00.078537 31752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } }
I20260806 12:57:00.078774 31759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } } }
I20260806 12:57:00.078822 31752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:57:00.078876 31759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: This master's current role is: FOLLOWER
W20260806 12:57:00.080374 31771 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:57:00.080440 31771 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:57:00.081974 31683 master_runner.cc:428] Detected that this master 7ae452e605104e909ae0c5ca954c82ba is joining an existing cluster
I20260806 12:57:00.082034 31683 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.83.61:40135
I20260806 12:57:00.089232 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.082989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53094 (local address 127.24.83.62:35127)
0806 12:57:00.083034 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.083035 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.083329 (+ 294us) server_negotiation.cc:407] Connection header received
0806 12:57:00.083393 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.083394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.083402 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.083467 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.083861 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.084487 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.085308 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.085572 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.085649 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.086581 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.086583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.086583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.086590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.088056 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.088409 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.088411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.088411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.088470 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.088687 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.088688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.088688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.088801 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.088857 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.089060 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.089139 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 12:57:00.089555 31778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.082939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:53094)
0806 12:57:00.083263 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.083272 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.083331 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.083473 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.083474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.083690 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.083786 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.083789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.084559 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.084560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.085222 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.085228 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.085669 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.085670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.085731 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.086130 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.086135 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.086507 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.088132 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.088133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.088133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.088337 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.088526 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.088527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.088528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.088589 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.088868 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.088869 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.088913 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.089354 (+ 441us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.089453 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":76,"threads_started":1}
I20260806 12:57:00.091169 31757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.083181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58926 (local address 127.24.83.61:40135)
0806 12:57:00.083223 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.083225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.083464 (+ 239us) server_negotiation.cc:407] Connection header received
0806 12:57:00.083525 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.083526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.083536 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.083586 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.083997 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.084697 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.085691 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.085990 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.086029 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.088267 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.088268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.088269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.088276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.090057 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.090363 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.090364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.090365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.090424 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.090615 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.090616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.090616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.090729 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.090794 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.090951 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.091076 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":25}
I20260806 12:57:00.091475 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.083142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:40135 (local address 127.0.0.1:58926)
0806 12:57:00.083419 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.083427 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.083480 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.083591 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.083592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.083816 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.083873 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.083875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.084784 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.084785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.085601 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.085604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.086091 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.086092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.086150 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.086861 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.086866 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.088184 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.090143 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.090144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.090144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.090299 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.090483 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.090484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.090485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.090550 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.090801 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.090801 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.090836 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.090928 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.091025 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":65,"threads_started":1}
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.62@KRBTEST.COM for oryx/127.24.83.61@KRBTEST.COM
I20260806 12:57:00.099038 31757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.092566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58942 (local address 127.24.83.61:40135)
0806 12:57:00.092602 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.092604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.092754 (+ 150us) server_negotiation.cc:407] Connection header received
0806 12:57:00.092850 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.092851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.092860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.092917 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.093297 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.093731 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.094465 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.094661 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.094730 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.096755 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.096756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.096756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.096762 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.097964 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.098397 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.098398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.098398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.098441 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.098596 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.098597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.098597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.098680 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.098726 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.098935 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.098988 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 12:57:00.099164 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.092516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:40135 (local address 127.0.0.1:58942)
0806 12:57:00.092656 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.092691 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.092776 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.092924 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.092928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.093180 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.093222 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.093224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.093809 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.093810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.094378 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.094380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.094731 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.094732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.094810 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.095251 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.095255 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.096675 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.098038 (+ 1363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.098064 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.098066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.098326 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.098482 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.098482 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.098483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.098543 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.098741 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.098748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.098806 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.098887 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.099001 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":59,"threads_started":1}
I20260806 12:57:00.099707 31199 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.83.61:40135: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba"
member_type: NON_VOTER
last_known_addr {
host: "127.24.83.61"
port: 40135
}
attrs {
promote: true
}
}
dest_uuid: "70b5b5dd010144199787f551a70ea8a6"
cas_config_opid_index: -1
I20260806 12:57:00.100096 31199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } }
I20260806 12:57:00.100556 31781 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7ae452e605104e909ae0c5ca954c82ba (127.24.83.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } }
I20260806 12:57:00.100955 31725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:00.101540 31781 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.83.61:40135
I20260806 12:57:00.101696 31725 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Refusing update from remote peer 70b5b5dd010144199787f551a70ea8a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 12:57:00.101809 31683 master.cc:537] Master@127.24.83.61:40135 shutting down...
I20260806 12:57:00.101769 31780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } }
I20260806 12:57:00.101903 31780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:00.102470 31780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 12:57:00.103807 31184 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 12:57:00.104260 31184 consensus_peers.cc:487] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 -> Peer 7ae452e605104e909ae0c5ca954c82ba (127.24.83.61:40135): Couldn't send request to peer 7ae452e605104e909ae0c5ca954c82ba. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 12:57:00.104979 31683 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:00.105065 31683 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:00.105113 31683 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: stopping tablet replica
I20260806 12:57:00.117853 31683 master.cc:552] Master@127.24.83.61:40135 shutdown complete.
I20260806 12:57:00.117902 31683 master_runner.cc:305] Clearing existing system tablet
I20260806 12:57:00.118048 31683 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:00.118095 31683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:00.118813 31683 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:00.119324 31786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.119453 31683 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:00.119514 31683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.119539 31683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/wal
uuid: "7ae452e605104e909ae0c5ca954c82ba"
format_stamp: "Formatted at 2026-08-06 12:57:00 on dist-test-slave-zkpd"
server_key: "b4b4edc19b4e274469f7eaaa91a52e4a"
server_key_iv: "771ee494c6e333cb3728ef22dda31ef9"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.120465 31683 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:00.122051 31683 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 12:57:00.122123 31683 log.cc:1196] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 12:57:00.122336 31683 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: Deleting consensus metadata
I20260806 12:57:00.122458 31683 master_runner.cc:315] Copying system tablet from 127.24.83.62:35127
I20260806 12:57:00.123067 31683 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: tablet copy: Beginning tablet copy session from remote peer at address 127.24.83.62:35127
I20260806 12:57:00.129276 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.123368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:53098)
0806 12:57:00.123490 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.123500 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.123559 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.123719 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.123720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.123870 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.123982 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.123984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.124916 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.124917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.125452 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.125454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.125912 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.125912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.125958 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.126428 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.126431 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.126708 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.128413 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.128413 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.128414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.128576 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.128769 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.128770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.128770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.128818 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.129054 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.129054 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.129089 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.129135 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.129206 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":87,"threads_started":1}
I20260806 12:57:00.129338 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.123448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53098 (local address 127.24.83.62:35127)
0806 12:57:00.123501 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.123503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.123546 (+ 43us) server_negotiation.cc:407] Connection header received
0806 12:57:00.123610 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.123611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.123625 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.123717 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.124056 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.124834 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.125525 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.125837 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.125908 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.126799 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.126801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.126801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.126808 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.128340 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.128641 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.128642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.128643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.128702 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.128870 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.128871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.128871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.128996 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.129064 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.129172 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.129246 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 12:57:00.129931 31220 tablet_copy_service.cc:140] P 70b5b5dd010144199787f551a70ea8a6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7ae452e605104e909ae0c5ca954c82ba ({username='oryx', principal='oryx/127.24.83.61@KRBTEST.COM'} at 127.0.0.1:53098)
I20260806 12:57:00.130057 31220 tablet_copy_service.cc:161] P 70b5b5dd010144199787f551a70ea8a6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7ae452e605104e909ae0c5ca954c82ba at {username='oryx', principal='oryx/127.24.83.61@KRBTEST.COM'} at 127.0.0.1:53098: session id = 7ae452e605104e909ae0c5ca954c82ba-00000000000000000000000000000000
I20260806 12:57:00.130752 31220 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 12:57:00.131354 31683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 12:57:00.132644 31683 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: tablet copy: Starting download of 0 data blocks...
I20260806 12:57:00.132810 31683 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: tablet copy: Starting download of 1 WAL segments...
I20260806 12:57:00.133713 31683 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 12:57:00.134601 31220 tablet_copy_service.cc:342] P 70b5b5dd010144199787f551a70ea8a6: Request end of tablet copy session 7ae452e605104e909ae0c5ca954c82ba-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.83.61@KRBTEST.COM'} at 127.0.0.1:53098
I20260806 12:57:00.134660 31220 tablet_copy_service.cc:433] P 70b5b5dd010144199787f551a70ea8a6: ending tablet copy session 7ae452e605104e909ae0c5ca954c82ba-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7ae452e605104e909ae0c5ca954c82ba
I20260806 12:57:00.137385 31683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:00.138513 31795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:00.138547 31794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:00.140858 31797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:00.141006 31683 server_base.cc:1034] running on GCE node
I20260806 12:57:00.141148 31683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:00.141304 31683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:00.142421 31683 hybrid_clock.cc:648] HybridClock initialized: now 1786021020142412 us; error 25 us; skew 500 ppm
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:00.143723 31683 init.cc:377] Logged in from keytab as oryx/127.24.83.61@KRBTEST.COM (short username oryx)
I20260806 12:57:00.144013 31683 webserver.cc:460] Webserver started at http://127.24.83.61:42703/ using document root <none> and password file <none>
I20260806 12:57:00.144142 31683 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:00.144186 31683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:00.144860 31683 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:00.145418 31804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.145532 31683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:00.145579 31683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/wal
uuid: "7ae452e605104e909ae0c5ca954c82ba"
format_stamp: "Formatted at 2026-08-06 12:57:00 on dist-test-slave-zkpd"
server_key: "b4b4edc19b4e274469f7eaaa91a52e4a"
server_key_iv: "771ee494c6e333cb3728ef22dda31ef9"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.145638 31683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:00.161713 31683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:00.163252 31683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:00.166425 31683 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.61:40135
I20260806 12:57:00.166441 31856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.61:40135 every 8 connection(s)
I20260806 12:57:00.166819 31683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
I20260806 12:57:00.167752 31857 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:57:00.168366 31857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: Bootstrap starting.
I20260806 12:57:00.171482 31857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:57:00.171728 31857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: Bootstrap complete.
I20260806 12:57:00.171969 31857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } }
I20260806 12:57:00.172061 31857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7ae452e605104e909ae0c5ca954c82ba, State: Initialized, Role: LEARNER
I20260806 12:57:00.172101 31857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } }
I20260806 12:57:00.172298 31857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:57:00.172281 31861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } }
I20260806 12:57:00.172477 31861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: This master's current role is: LEARNER
I20260806 12:57:00.173846 31872 catalog_manager.cc:1259] Loaded cluster ID: c298d5eafbc745d6945d505298714ac0
I20260806 12:57:00.173933 31872 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: loading cluster ID for follower catalog manager: success
I20260806 12:57:00.175189 31872 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: acquiring CA information for follower catalog manager: success
I20260806 12:57:00.175585 31872 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 12:57:00.178298 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.170702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58960 (local address 127.24.83.61:40135)
0806 12:57:00.170836 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.170838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.170846 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:00.170869 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.170870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.170879 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.170959 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.171502 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.172503 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.173383 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.173689 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.173759 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.175220 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.175222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.175223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.175230 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.177187 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.177518 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.177519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.177520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.177589 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.177776 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.177777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.177778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.177921 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.178021 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.178108 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.178206 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.24.83.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.60.keytab.
Entry for principal oryx/127.24.83.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.60.keytab.
I20260806 12:57:00.203192 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.60:42009
--webserver_interface=127.24.83.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.60.keytab
--principal=oryx/127.24.83.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:35127,127.24.83.61:40135,127.24.83.60:42009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:00.290845 31875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:00.291055 31875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:00.291079 31875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:00.292225 31875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:57:00.292263 31875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:00.292276 31875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:00.292289 31875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:57:00.292301 31875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:57:00.293989 31875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:35127,127.24.83.61:40135,127.24.83.60:42009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.60.keytab
--principal=oryx/127.24.83.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.60:42009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.83.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:00.294199 31875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:00.294377 31875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:00.299805 31883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:00.299856 31881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:00.299968 31875 server_base.cc:1034] running on GCE node
W20260806 12:57:00.300055 31880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:00.300328 31875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:00.300643 31875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:00.301805 31875 hybrid_clock.cc:648] HybridClock initialized: now 1786021020301795 us; error 27 us; skew 500 ppm
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:00.304785 31875 init.cc:377] Logged in from keytab as oryx/127.24.83.60@KRBTEST.COM (short username oryx)
I20260806 12:57:00.305230 31875 webserver.cc:460] Webserver started at http://127.24.83.60:36891/ using document root <none> and password file <none>
I20260806 12:57:00.305462 31875 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:00.305528 31875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:00.305675 31875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:00.306830 31875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data/instance:
uuid: "63f73a41780e4913878e30fbdea6bc54"
format_stamp: "Formatted at 2026-08-06 12:57:00 on dist-test-slave-zkpd"
server_key: "bb0a74444f6550a2bc583331da5e150e"
server_key_iv: "777f84d00cba9b4d26fe1948b0e4f8a0"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.307217 31875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/wal/instance:
uuid: "63f73a41780e4913878e30fbdea6bc54"
format_stamp: "Formatted at 2026-08-06 12:57:00 on dist-test-slave-zkpd"
server_key: "bb0a74444f6550a2bc583331da5e150e"
server_key_iv: "777f84d00cba9b4d26fe1948b0e4f8a0"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.308959 31875 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:57:00.309731 31890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.309944 31875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:00.310042 31875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/wal
uuid: "63f73a41780e4913878e30fbdea6bc54"
format_stamp: "Formatted at 2026-08-06 12:57:00 on dist-test-slave-zkpd"
server_key: "bb0a74444f6550a2bc583331da5e150e"
server_key_iv: "777f84d00cba9b4d26fe1948b0e4f8a0"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.310122 31875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:00.322623 31875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:00.325117 31875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:00.325294 31875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:00.333735 31875 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.60:42009
I20260806 12:57:00.333853 31946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.60:42009 every 8 connection(s)
I20260806 12:57:00.334203 31875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
I20260806 12:57:00.335417 31947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:00.337525 31947 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:57:00.338369 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 31875
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.60@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.83.60@KRBTEST.COM
I20260806 12:57:00.348106 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.338332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53108 (local address 127.24.83.62:35127)
0806 12:57:00.338392 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.338394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.338709 (+ 315us) server_negotiation.cc:407] Connection header received
0806 12:57:00.339585 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.339587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.339600 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.339654 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.340291 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.340782 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.341776 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.341970 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.342051 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.344814 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.344816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.344816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.344823 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.346515 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.347136 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.347137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.347138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.347192 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.347475 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.347475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.347475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.347574 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.347623 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.347984 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.348045 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 12:57:00.348444 31949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.338193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:53108)
0806 12:57:00.338532 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.338588 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.339475 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.339672 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.339675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.339988 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.340196 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.340201 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.340914 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.340918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.341685 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.341687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.342179 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.342180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.342316 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.342954 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.342964 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.344693 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.346635 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.346643 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.346645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.347055 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.347260 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.347261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.347262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.347404 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.347638 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.347644 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.347814 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.347973 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.348141 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":193,"threads_started":1}
I20260806 12:57:00.349165 31947 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:57:00.349568 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.338921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.24.83.60:42009)
0806 12:57:00.339128 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.339132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.339161 (+ 29us) server_negotiation.cc:407] Connection header received
0806 12:57:00.339481 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.339490 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.339510 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.339607 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.340267 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.341295 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.341994 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.342355 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.342451 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.345404 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.345412 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.345412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.345420 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.348013 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.348437 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.348440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.348440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.348543 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.348775 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.348776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.348776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.348963 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.349107 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.349191 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.349275 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":101,"threads_started":1}
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.60@KRBTEST.COM for oryx/127.24.83.61@KRBTEST.COM
I20260806 12:57:00.355288 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.349464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58974 (local address 127.24.83.61:40135)
0806 12:57:00.349523 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.349525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.349560 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:57:00.349632 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.349633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.349643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.349687 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.349975 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.350437 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.351077 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.351288 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.351362 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.352993 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.352994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.352994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.353001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.354238 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.354556 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.354557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.354557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.354614 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.354805 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.354806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.354806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.354912 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.354985 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.355125 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.355202 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 12:57:00.355276 31949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.349400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:40135 (local address 127.0.0.1:58974)
0806 12:57:00.349507 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.349530 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.349573 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.349726 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.349727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.349857 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.349898 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.349899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.350522 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.350523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.350991 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.350992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.351364 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.351365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.351423 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.351760 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.351764 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.352902 (+ 1138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.354324 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.354325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.354325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.354486 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.354678 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.354679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.354679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.354733 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.354977 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.354977 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.355018 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.355100 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.355188 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 12:57:00.355815 31947 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.60@KRBTEST.COM for oryx/127.24.83.60@KRBTEST.COM
I20260806 12:57:00.361654 31949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.356064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42009 (local address 127.0.0.1:34052)
0806 12:57:00.356114 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.356125 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.356171 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.356271 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.356272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.356409 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.356437 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.356438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.357006 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.357007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.357561 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.357563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.357895 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.357896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.357932 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.358255 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.358259 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.359335 (+ 1076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.360847 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.360848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.360848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.361006 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.361162 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.361163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.361163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.361214 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.361403 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.361404 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.361448 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.361507 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.361578 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 12:57:00.361703 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.356126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34052 (local address 127.24.83.60:42009)
0806 12:57:00.356173 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.356174 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.356181 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:57:00.356219 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.356220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.356227 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.356267 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.356489 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.356939 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.357636 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.357837 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.357885 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.359415 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.359416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.359416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.359422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.360779 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.361067 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.361067 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.361068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.361110 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.361260 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.361261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.361261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.361342 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.361382 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.361523 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.361622 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":31}
I20260806 12:57:00.363504 31947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: Bootstrap starting.
I20260806 12:57:00.364622 31947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:00.365156 31947 log.cc:824] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:00.365931 31947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: No bootstrap required, opened a new log
I20260806 12:57:00.368841 31947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } }
I20260806 12:57:00.368963 31947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:00.368986 31947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f73a41780e4913878e30fbdea6bc54, State: Initialized, Role: FOLLOWER
I20260806 12:57:00.369107 31947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } }
I20260806 12:57:00.369449 31947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:57:00.369503 31953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } } }
I20260806 12:57:00.369639 31953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 12:57:00.371765 31964 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:57:00.371825 31964 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:57:00.374104 31875 master_runner.cc:428] Detected that this master 63f73a41780e4913878e30fbdea6bc54 is joining an existing cluster
I20260806 12:57:00.374167 31875 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.83.60:42009
I20260806 12:57:00.383369 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.375394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.24.83.62:35127)
0806 12:57:00.375434 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.375436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.375803 (+ 367us) server_negotiation.cc:407] Connection header received
0806 12:57:00.375932 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.375933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.375945 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.375997 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.376788 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.377284 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.378615 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.378822 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.378898 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.380283 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.380284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.380285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.380291 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.381962 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.382318 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.382320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.382320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.382389 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.382736 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.382737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.382738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.382908 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.382979 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.383179 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.383255 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 12:57:00.383872 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.375368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:41766)
0806 12:57:00.375722 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.375746 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.375828 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.376046 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.376047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.376420 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.376676 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.376679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.377358 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.377358 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.378535 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.378538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.378982 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.378982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.379060 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.379698 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.379702 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.380183 (+ 481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.382030 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.382031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.382031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.382252 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.382475 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.382476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.382477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.382617 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.383009 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.383010 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.383059 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.383456 (+ 397us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.383573 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":255,"threads_started":1}
I20260806 12:57:00.384819 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.375970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38230 (local address 127.24.83.61:40135)
0806 12:57:00.376013 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.376015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.376689 (+ 674us) server_negotiation.cc:407] Connection header received
0806 12:57:00.376743 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.376744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.376756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.376808 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.377426 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.378236 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.380034 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.380332 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.380409 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.381762 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.381763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.381764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.381771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.383463 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.384015 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.384016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.384017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.384087 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.384277 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.384277 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.384278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.384378 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.384453 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.384665 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.384733 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:57:00.384889 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.375823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:40135 (local address 127.0.0.1:38230)
0806 12:57:00.376606 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.376617 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.376682 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.376860 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.376860 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.377144 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.377320 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.377322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.378326 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.378327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.379954 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.379957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.380434 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.380435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.380859 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.381275 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.381279 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.381665 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.383585 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.383586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.383586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.383854 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.384164 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.384165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.384165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.384222 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.384446 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.384447 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.384557 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.384632 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.384729 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":175,"threads_started":1}
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021019, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.62@KRBTEST.COM for oryx/127.24.83.60@KRBTEST.COM
I20260806 12:57:00.400952 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 31263
I20260806 12:57:00.407052 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:37175
--local_ip_for_outbound_sockets=127.24.83.1
--tserver_master_addrs=127.24.83.62:35127
--webserver_port=43873
--webserver_interface=127.24.83.1
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab
--principal=oryx/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.83.62:35127,127.24.83.61:40135
--tserver_master_addrs=127.24.83.62:35127,127.24.83.61:40135,127.24.83.60:42009 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:57:00.407274 31953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:57:00.407379 31953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } }
I20260806 12:57:00.407644 31953 leader_election.cc:288] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127), 7ae452e605104e909ae0c5ca954c82ba (127.24.83.61:40135)
I20260806 12:57:00.407968 31831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63f73a41780e4913878e30fbdea6bc54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ae452e605104e909ae0c5ca954c82ba" is_pre_election: true
I20260806 12:57:00.408104 31831 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 LEARNER]: Handling vote request from an unknown peer 63f73a41780e4913878e30fbdea6bc54
I20260806 12:57:00.408161 31831 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 63f73a41780e4913878e30fbdea6bc54 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260806 12:57:00.407861 31210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63f73a41780e4913878e30fbdea6bc54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70b5b5dd010144199787f551a70ea8a6" is_pre_election: true
I20260806 12:57:00.408665 31891 leader_election.cc:302] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63f73a41780e4913878e30fbdea6bc54; no voters: 70b5b5dd010144199787f551a70ea8a6, 7ae452e605104e909ae0c5ca954c82ba
I20260806 12:57:00.409423 31953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:00.411450 31953 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 12:57:00.415267 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.376203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34058 (local address 127.24.83.60:42009)
0806 12:57:00.376254 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.376256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.384197 (+ 7941us) server_negotiation.cc:407] Connection header received
0806 12:57:00.384222 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.384223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.384234 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.384287 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.386812 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.387345 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.388617 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.388974 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.389020 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.410798 (+ 21778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.410799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.410800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.410805 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.412632 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.413397 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.413398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.413399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.413454 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.414027 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.414028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.414029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.414189 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.414247 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.415009 (+ 762us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.415147 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":30}
I20260806 12:57:00.415421 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.376347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42009 (local address 127.0.0.1:34058)
0806 12:57:00.383994 (+ 7647us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.384004 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.384140 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.386121 (+ 1981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.386122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.386321 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.386704 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.386708 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.387432 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.387433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.388443 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.388446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.389091 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.389092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.389240 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.409119 (+ 19879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.409126 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.410691 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.412726 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.412727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.412727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.412938 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.413522 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.413523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.413523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.413584 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.414427 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.414428 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.414476 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.414942 (+ 466us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.415069 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7635,"thread_start_us":163,"threads_started":1}
I20260806 12:57:00.415555 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.386367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42009 (local address 127.0.0.1:34064)
0806 12:57:00.386439 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.386452 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.386502 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.387435 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.387436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.387556 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.387590 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.387591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.388544 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.388545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.389062 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.389063 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.389552 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.389553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.389622 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.390075 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.390079 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.391545 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.413663 (+ 22118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.413665 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.413665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.413932 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.414575 (+ 643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.414576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.414577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.414650 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.415213 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.415214 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.415275 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.415328 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.415458 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 12:57:00.416155 31973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.386405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34064 (local address 127.24.83.60:42009)
0806 12:57:00.386679 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.386681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.386689 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:00.386728 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.386729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.386737 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.386817 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.387701 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.388449 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.389140 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.389475 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.389552 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.391711 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.391712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.391713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.391718 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.413287 (+ 21569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.414293 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.414294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.414294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.414344 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.414730 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.414731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.414731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.414859 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.415763 (+ 904us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.415804 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.415883 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":209,"threads_started":1}
I20260806 12:57:00.416628 31199 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.83.60:42009: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "63f73a41780e4913878e30fbdea6bc54"
member_type: NON_VOTER
last_known_addr {
host: "127.24.83.60"
port: 42009
}
attrs {
promote: true
}
}
dest_uuid: "70b5b5dd010144199787f551a70ea8a6"
cas_config_opid_index: 5
I20260806 12:57:00.416855 31199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } }
I20260806 12:57:00.417228 31781 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 63f73a41780e4913878e30fbdea6bc54 (127.24.83.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.417577 31919 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 FOLLOWER]: Refusing update from remote peer 70b5b5dd010144199787f551a70ea8a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 12:57:00.417821 31780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:00.418526 31781 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.83.60:42009
I20260806 12:57:00.418622 31781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.418718 31781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:00.419430 31919 raft_consensus.cc:858] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } }
W20260806 12:57:00.419523 31919 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } }
I20260806 12:57:00.419694 31875 master.cc:537] Master@127.24.83.60:42009 shutting down...
I20260806 12:57:00.420099 31919 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7ae452e605104e909ae0c5ca954c82ba (127.24.83.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } }
I20260806 12:57:00.421144 31978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } }
I20260806 12:57:00.421247 31978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 12:57:00.421290 31978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } }
I20260806 12:57:00.421339 31978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 12:57:00.421501 31953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70b5b5dd010144199787f551a70ea8a6. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } }
I20260806 12:57:00.421573 31953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 12:57:00.423717 31975 mvcc.cc:205] Tried to move back new op lower bound from 7315542095597428736 to 7315542095497744384. Current Snapshot: MvccSnapshot[applied={T|T < 7315542095507484672}]
I20260806 12:57:00.424463 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.418911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:40135 (local address 127.0.0.1:38238)
0806 12:57:00.419648 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.419659 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.419704 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.419953 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.419954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.419956 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:00.420002 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.420003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.421874 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.421875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.423144 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.423147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.424162 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.424163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.424234 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.424275 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.424337 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.424385 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720}
I20260806 12:57:00.424860 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.419204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38238 (local address 127.24.83.61:40135)
0806 12:57:00.419316 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.419319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.419758 (+ 439us) server_negotiation.cc:407] Connection header received
0806 12:57:00.419787 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.419788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.419810 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.419870 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:00.420086 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.420827 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.423251 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.423853 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.424047 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.424130 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.424729 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.424783 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 12:57:00.424831 31780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.424914 31780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:00.425132 31831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 LEARNER]: Refusing update from remote peer 70b5b5dd010144199787f551a70ea8a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 12:57:00.425351 31781 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260806 12:57:00.425587 31187 consensus_peers.cc:487] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 -> Peer 63f73a41780e4913878e30fbdea6bc54 (127.24.83.60:42009): Couldn't send request to peer 63f73a41780e4913878e30fbdea6bc54. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 12:57:00.425860 31875 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260806 12:57:00.425938 31875 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260806 12:57:00.425848 31831 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 63f73a41780e4913878e30fbdea6bc54 (127.24.83.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.425974 31875 tablet_replica.cc:330] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: stopping tablet replica
I20260806 12:57:00.426893 31861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70b5b5dd010144199787f551a70ea8a6. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.426980 31861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: This master's current role is: LEARNER
I20260806 12:57:00.426913 31983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: NON_VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: true } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.427022 31983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: This master's current role is: LEARNER
I20260806 12:57:00.432333 31780 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6: attempting to promote NON_VOTER 7ae452e605104e909ae0c5ca954c82ba to VOTER
I20260806 12:57:00.432549 31780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } }
I20260806 12:57:00.433068 31831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 LEARNER]: Refusing update from remote peer 70b5b5dd010144199787f551a70ea8a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 12:57:00.433095 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.432979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42009 (local address 127.0.0.1:34072)
0806 12:57:00.433015 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.433061 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 12:57:00.433259 31781 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 12:57:00.433492 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.433414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42009 (local address 127.0.0.1:34074)
0806 12:57:00.433449 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.433470 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 12:57:00.433676 31187 consensus_peers.cc:487] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 -> Peer 63f73a41780e4913878e30fbdea6bc54 (127.24.83.60:42009): Couldn't send request to peer 63f73a41780e4913878e30fbdea6bc54. Status: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42009: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 12:57:00.434051 31781 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7ae452e605104e909ae0c5ca954c82ba (127.24.83.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.434762 31781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.434808 31780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.434847 31781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:00.434859 31780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:00.434986 31831 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7ae452e605104e909ae0c5ca954c82ba (127.24.83.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.435714 31983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.435942 31983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:57:00.439780 31875 master.cc:552] Master@127.24.83.60:42009 shutdown complete.
I20260806 12:57:00.439849 31875 master_runner.cc:305] Clearing existing system tablet
I20260806 12:57:00.440047 31875 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:00.440109 31875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:00.440701 31875 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:57:00.441186 31986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.441355 31875 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:00.441416 31875 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.441447 31875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/wal
uuid: "63f73a41780e4913878e30fbdea6bc54"
format_stamp: "Formatted at 2026-08-06 12:57:00 on dist-test-slave-zkpd"
server_key: "bb0a74444f6550a2bc583331da5e150e"
server_key_iv: "777f84d00cba9b4d26fe1948b0e4f8a0"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.442108 31875 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:00.443342 31875 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 12:57:00.443418 31875 log.cc:1196] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 12:57:00.443650 31875 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: Deleting consensus metadata
I20260806 12:57:00.443778 31875 master_runner.cc:315] Copying system tablet from 127.24.83.62:35127
I20260806 12:57:00.444420 31875 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: tablet copy: Beginning tablet copy session from remote peer at address 127.24.83.62:35127
I20260806 12:57:00.450548 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.444744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41778 (local address 127.24.83.62:35127)
0806 12:57:00.444797 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.444800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.444920 (+ 120us) server_negotiation.cc:407] Connection header received
0806 12:57:00.444997 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.444998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.445009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.445065 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.445437 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.445993 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.446747 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.446953 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.447035 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.448012 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.448013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.448013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.448020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.449439 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.449793 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.449794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.449795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.449851 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.450051 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.450052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.450052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.450161 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.450228 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.450391 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.450463 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:57:00.450564 31992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.444715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:41778)
0806 12:57:00.444848 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.444860 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.444936 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.445070 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.445071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.445290 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.445354 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.445356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.446066 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.446067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.446668 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.446670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.447036 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.447037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.447102 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.447577 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.447582 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.447922 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.449537 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.449538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.449539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.449712 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.449925 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.449926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.449926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.449986 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.450234 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.450235 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.450278 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.450350 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.450456 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260806 12:57:00.451112 31220 tablet_copy_service.cc:140] P 70b5b5dd010144199787f551a70ea8a6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 63f73a41780e4913878e30fbdea6bc54 ({username='oryx', principal='oryx/127.24.83.60@KRBTEST.COM'} at 127.0.0.1:41778)
I20260806 12:57:00.451170 31220 tablet_copy_service.cc:161] P 70b5b5dd010144199787f551a70ea8a6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 63f73a41780e4913878e30fbdea6bc54 at {username='oryx', principal='oryx/127.24.83.60@KRBTEST.COM'} at 127.0.0.1:41778: session id = 63f73a41780e4913878e30fbdea6bc54-00000000000000000000000000000000
I20260806 12:57:00.451606 31220 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 12:57:00.452153 31875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 12:57:00.453181 31875 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: tablet copy: Starting download of 0 data blocks...
I20260806 12:57:00.453339 31875 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: tablet copy: Starting download of 1 WAL segments...
I20260806 12:57:00.454082 31875 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 12:57:00.454770 31220 tablet_copy_service.cc:342] P 70b5b5dd010144199787f551a70ea8a6: Request end of tablet copy session 63f73a41780e4913878e30fbdea6bc54-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.83.60@KRBTEST.COM'} at 127.0.0.1:41778
I20260806 12:57:00.454823 31220 tablet_copy_service.cc:433] P 70b5b5dd010144199787f551a70ea8a6: ending tablet copy session 63f73a41780e4913878e30fbdea6bc54-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 63f73a41780e4913878e30fbdea6bc54
I20260806 12:57:00.457216 31875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:00.458434 31996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:00.458451 31993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:00.460057 31875 server_base.cc:1034] running on GCE node
W20260806 12:57:00.460170 31994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:00.460297 31875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:00.460448 31875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:00.461572 31875 hybrid_clock.cc:648] HybridClock initialized: now 1786021020461562 us; error 29 us; skew 500 ppm
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:00.462920 31875 init.cc:377] Logged in from keytab as oryx/127.24.83.60@KRBTEST.COM (short username oryx)
I20260806 12:57:00.463199 31875 webserver.cc:460] Webserver started at http://127.24.83.60:39451/ using document root <none> and password file <none>
I20260806 12:57:00.463316 31875 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:00.463351 31875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:00.463951 31875 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:00.464536 32003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.464655 31875 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.464694 31875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/wal
uuid: "63f73a41780e4913878e30fbdea6bc54"
format_stamp: "Formatted at 2026-08-06 12:57:00 on dist-test-slave-zkpd"
server_key: "bb0a74444f6550a2bc583331da5e150e"
server_key_iv: "777f84d00cba9b4d26fe1948b0e4f8a0"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.464751 31875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:00.486934 31875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:00.488696 31875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:00.492501 31875 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.60:42009
I20260806 12:57:00.492623 32057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.60:42009 every 8 connection(s)
I20260806 12:57:00.492846 31875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
I20260806 12:57:00.493642 32058 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:57:00.494055 32058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: Bootstrap starting.
I20260806 12:57:00.496057 32058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:57:00.496290 32058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: Bootstrap complete.
I20260806 12:57:00.496479 32058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } }
I20260806 12:57:00.496563 32058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 63f73a41780e4913878e30fbdea6bc54, State: Initialized, Role: LEARNER
I20260806 12:57:00.496609 32058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } }
I20260806 12:57:00.496832 32058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:57:00.497087 32061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.497251 32061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: This master's current role is: LEARNER
I20260806 12:57:00.498531 32072 catalog_manager.cc:1259] Loaded cluster ID: c298d5eafbc745d6945d505298714ac0
I20260806 12:57:00.498577 32072 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: loading cluster ID for follower catalog manager: success
I20260806 12:57:00.499562 32072 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: acquiring CA information for follower catalog manager: success
I20260806 12:57:00.499840 32072 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 12:57:00.533749 31974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:00.534034 31974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:00.534077 31974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:00.535748 31974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:00.535805 31974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:00.535844 31974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:57:00.537922 31974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab
--principal=oryx/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:37175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=43873
--tserver_master_addrs=127.24.83.62:35127,127.24.83.61:40135,127.24.83.60:42009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:00.538156 31974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:00.538398 31974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:00.543970 32080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:00.543927 32077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:00.543931 32078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:00.544422 31974 server_base.cc:1034] running on GCE node
I20260806 12:57:00.544622 31974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:00.544906 31974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:00.546087 31974 hybrid_clock.cc:648] HybridClock initialized: now 1786021020546068 us; error 40 us; skew 500 ppm
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:00.548976 31974 init.cc:377] Logged in from keytab as oryx/127.24.83.1@KRBTEST.COM (short username oryx)
I20260806 12:57:00.549420 31974 webserver.cc:460] Webserver started at http://127.24.83.1:43873/ using document root <none> and password file <none>
I20260806 12:57:00.549634 31974 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:00.549713 31974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:00.551101 31974 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:00.551800 32087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.552016 31974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:00.552116 31974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "a30d865941464d91bd03c5cc1cb2bee2"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "74e5eae5802962e79de7bcca0d71eeb8"
server_key_iv: "5ed3f7406ac32597ab3d2451b5fae959"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.552448 31974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:00.570813 31974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:00.572901 31974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:00.573081 31974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:00.573519 31974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:00.573890 31974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:00.573925 31974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.573973 31974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:00.573997 31974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.585966 31974 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:37175
I20260806 12:57:00.586117 32207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:37175 every 8 connection(s)
I20260806 12:57:00.586402 31974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:57:00.593840 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 31974
I20260806 12:57:00.594025 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 31403
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.60@KRBTEST.COM
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.61@KRBTEST.COM
I20260806 12:57:00.600366 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.588103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:33087 (local address 127.24.83.61:40135)
0806 12:57:00.588175 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.588177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.588188 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:57:00.589498 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.589499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.589511 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.589583 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.590222 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.590931 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.592433 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.592713 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.592781 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.596586 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.596588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.596588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.596596 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.598526 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.599281 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.599282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.599283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.599334 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.599675 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.599676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.599677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.599803 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.599879 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.600185 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.600278 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:57:00.600474 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.587322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:42449 (local address 127.24.83.60:42009)
0806 12:57:00.587514 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.587516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.587803 (+ 287us) server_negotiation.cc:407] Connection header received
0806 12:57:00.589163 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.589165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.589180 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.589302 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.590150 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.591130 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.592262 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.592541 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.592606 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.596516 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.596518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.596518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.596526 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.598799 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.599250 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.599251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.599251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.599319 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.599680 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.599682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.599682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.599876 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.599970 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.600208 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.600318 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":52,"threads_started":1}
I20260806 12:57:00.600647 32215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.587818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:40135 (local address 127.24.83.1:33087)
0806 12:57:00.588081 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.588093 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.589059 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.589616 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.589616 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.589882 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.590087 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.590090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.591013 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.591014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.592340 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.592343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.592795 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.592796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.592919 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.593523 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.593529 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.596494 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.598632 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.598642 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.598646 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.599165 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.599410 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.599410 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.599411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.599571 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.599871 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.599879 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.600048 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.600149 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.600313 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":40,"thread_start_us":132,"threads_started":1}
I20260806 12:57:00.600934 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.587168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42009 (local address 127.24.83.1:42449)
0806 12:57:00.587672 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.587707 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.589040 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.589385 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.589389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.589820 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.590051 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.590056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.591207 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.591208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.592178 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.592180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.592620 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.592621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.592809 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.593341 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.593349 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.596414 (+ 3065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.598878 (+ 2464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.598879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.598879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.599162 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.599392 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.599393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.599393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.599579 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.600011 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.600012 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.600065 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.600559 (+ 494us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.600686 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":18,"thread_start_us":282,"threads_started":1}
I20260806 12:57:00.601208 32210 heartbeater.cc:347] Connected to a master server at 127.24.83.61:40135
I20260806 12:57:00.601295 32210 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:00.601501 32210 heartbeater.cc:511] Master 127.24.83.61:40135 requested a full tablet report, sending...
I20260806 12:57:00.602020 31821 ts_manager.cc:194] Registered new tserver with Master: a30d865941464d91bd03c5cc1cb2bee2 (127.24.83.1:37175)
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:57:00.608587 32209 heartbeater.cc:347] Connected to a master server at 127.24.83.60:42009
I20260806 12:57:00.608645 32209 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:00.608721 32209 heartbeater.cc:511] Master 127.24.83.60:42009 requested a full tablet report, sending...
I20260806 12:57:00.609153 32020 ts_manager.cc:194] Registered new tserver with Master: a30d865941464d91bd03c5cc1cb2bee2 (127.24.83.1:37175)
I20260806 12:57:00.609232 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:37379
--local_ip_for_outbound_sockets=127.24.83.2
--tserver_master_addrs=127.24.83.62:35127
--webserver_port=36979
--webserver_interface=127.24.83.2
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab
--principal=oryx/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.83.62:35127,127.24.83.61:40135
--tserver_master_addrs=127.24.83.62:35127,127.24.83.61:40135,127.24.83.60:42009 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:57:00.609385 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.587314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:55157 (local address 127.24.83.62:35127)
0806 12:57:00.587368 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.587370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.587831 (+ 461us) server_negotiation.cc:407] Connection header received
0806 12:57:00.589362 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.589363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.589375 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.589438 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.590253 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.590733 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.592467 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.592723 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.592790 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.606214 (+ 13424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.606216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.606216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.606225 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.607978 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.608417 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.608419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.608420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.608493 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.608721 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.608723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.608723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.608911 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.608984 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.609167 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.609254 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:57:00.609937 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.587168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.1:55157)
0806 12:57:00.587649 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.587697 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.589055 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.589494 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.589494 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.589886 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.590153 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.590156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.590928 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.590929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.592365 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.592367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.592845 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.592846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.592950 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.604529 (+ 11579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.604534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.606118 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.608067 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.608068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.608068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.608300 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.608566 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.608568 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.608568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.608655 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.608988 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.608989 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.609050 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.609113 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.609737 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":58,"thread_start_us":177,"threads_started":1}
I20260806 12:57:00.610508 32208 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35127
I20260806 12:57:00.610579 32208 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:00.610662 32208 heartbeater.cc:511] Master 127.24.83.62:35127 requested a full tablet report, sending...
I20260806 12:57:00.610970 31199 ts_manager.cc:194] Re-registered known tserver with Master: a30d865941464d91bd03c5cc1cb2bee2 (127.24.83.1:37175)
I20260806 12:57:00.611526 31199 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:55157
W20260806 12:57:00.693579 32216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:00.693802 32216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:00.693826 32216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:00.694896 32216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:00.694929 32216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:00.694954 32216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:57:00.696427 32216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab
--principal=oryx/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:37379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=36979
--tserver_master_addrs=127.24.83.62:35127,127.24.83.61:40135,127.24.83.60:42009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:00.696643 32216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:00.696821 32216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:00.701714 32225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:00.701757 32222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:00.701912 32216 server_base.cc:1034] running on GCE node
W20260806 12:57:00.701774 32223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:00.702154 32216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:00.702375 32216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:00.703534 32216 hybrid_clock.cc:648] HybridClock initialized: now 1786021020703535 us; error 35 us; skew 500 ppm
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:00.705971 32216 init.cc:377] Logged in from keytab as oryx/127.24.83.2@KRBTEST.COM (short username oryx)
I20260806 12:57:00.706357 32216 webserver.cc:460] Webserver started at http://127.24.83.2:36979/ using document root <none> and password file <none>
I20260806 12:57:00.706561 32216 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:00.706637 32216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:00.708317 32216 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:57:00.708967 32233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.709139 32216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:57:00.709230 32216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "bdbf455301784493b51959a559bb65d2"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "8135dad94f97574a738f9de893a3d152"
server_key_iv: "9498e96daa3c70d86eabbfb81931ca87"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.709617 32216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:00.728335 32216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:00.730603 32216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:00.730758 32216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:00.731007 32216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:00.731304 32216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:00.731354 32216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.731406 32216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:00.731434 32216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.737537 32216 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:37379
I20260806 12:57:00.737587 32350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:37379 every 8 connection(s)
I20260806 12:57:00.737910 32216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:57:00.745069 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 32216
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.61@KRBTEST.COM
I20260806 12:57:00.745250 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 31544
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.60@KRBTEST.COM
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:57:00.750430 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.739073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:57021 (local address 127.24.83.62:35127)
0806 12:57:00.739127 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.739129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.739139 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:57:00.740161 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.740163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.740176 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.740211 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.741066 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.741582 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.742964 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.743242 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.743283 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.746974 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.746976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.746976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.746983 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.748774 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.749257 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.749259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.749259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.749335 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.749621 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.749622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.749623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.749769 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.749813 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.750185 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.750274 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 12:57:00.750967 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.738886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.2:57021)
0806 12:57:00.739031 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.739042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.740040 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.740384 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.740384 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.740770 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.740942 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.740948 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.741677 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.741677 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.742831 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.742834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.743360 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.743360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.743985 (+ 625us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.744474 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.744480 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.746871 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.748859 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.748865 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.748868 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.749163 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.749410 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.749410 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.749411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.749544 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.749861 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.749866 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.750029 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.750527 (+ 498us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.750709 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260806 12:57:00.751410 32351 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35127
I20260806 12:57:00.751498 32351 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:00.751693 32351 heartbeater.cc:511] Master 127.24.83.62:35127 requested a full tablet report, sending...
I20260806 12:57:00.752350 31199 ts_manager.cc:194] Re-registered known tserver with Master: bdbf455301784493b51959a559bb65d2 (127.24.83.2:37379)
I20260806 12:57:00.752836 31199 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:57021
I20260806 12:57:00.753916 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:40855
--local_ip_for_outbound_sockets=127.24.83.3
--tserver_master_addrs=127.24.83.62:35127
--webserver_port=45203
--webserver_interface=127.24.83.3
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab
--principal=oryx/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.83.62:35127,127.24.83.61:40135
--tserver_master_addrs=127.24.83.62:35127,127.24.83.61:40135,127.24.83.60:42009 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:57:00.754793 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.738581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:40135 (local address 127.24.83.2:59127)
0806 12:57:00.738748 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.738787 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.740039 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.740375 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.740378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.740744 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.740941 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.740945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.741954 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.741955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.742965 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.742967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.743355 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.743357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.743435 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.743977 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.743980 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.747639 (+ 3659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.752955 (+ 5316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.752956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.752957 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.753172 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.753405 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.753406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.753406 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.753457 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.754072 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.754073 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.754124 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.754493 (+ 369us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.754599 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
I20260806 12:57:00.754801 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.738563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:59127 (local address 127.24.83.61:40135)
0806 12:57:00.738617 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.738619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.738895 (+ 276us) server_negotiation.cc:407] Connection header received
0806 12:57:00.740174 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.740176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.740189 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.740221 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.741038 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.741848 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.743042 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.743271 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.743352 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.747750 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.747752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.747752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.747762 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.752790 (+ 5028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.753267 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.753268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.753268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.753328 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.753531 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.753532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.753532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.753962 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.754017 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.754549 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.754644 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:57:00.755703 32353 heartbeater.cc:347] Connected to a master server at 127.24.83.61:40135
I20260806 12:57:00.755759 32353 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:00.755842 32353 heartbeater.cc:511] Master 127.24.83.61:40135 requested a full tablet report, sending...
I20260806 12:57:00.756386 31821 ts_manager.cc:194] Registered new tserver with Master: bdbf455301784493b51959a559bb65d2 (127.24.83.2:37379)
I20260806 12:57:00.756747 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.738600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:57419 (local address 127.24.83.60:42009)
0806 12:57:00.738645 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.738648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.738890 (+ 242us) server_negotiation.cc:407] Connection header received
0806 12:57:00.740162 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.740163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.740179 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.740211 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.741101 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.741876 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.742789 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.743035 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.743107 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.753352 (+ 10245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.753353 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.753354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.753362 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.755141 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.755704 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.755705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.755705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.755769 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.756048 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.756049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.756049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.756187 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.756258 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.756551 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.756643 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:57:00.757109 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.738580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42009 (local address 127.24.83.2:57419)
0806 12:57:00.738747 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.738787 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.740036 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.740376 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.740379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.740762 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.740996 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.740998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.741988 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.741989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.742685 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.742687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.743113 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.743113 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.743239 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.743781 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.743791 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.753253 (+ 9462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.755420 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.755421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.755421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.755622 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.755851 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.755852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.755852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.755933 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.756259 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.756260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.756379 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.756833 (+ 454us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.756930 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":40,"thread_start_us":90,"threads_started":1}
I20260806 12:57:00.757750 32352 heartbeater.cc:347] Connected to a master server at 127.24.83.60:42009
I20260806 12:57:00.757827 32352 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:00.757875 32352 heartbeater.cc:511] Master 127.24.83.60:42009 requested a full tablet report, sending...
I20260806 12:57:00.758188 32020 ts_manager.cc:194] Registered new tserver with Master: bdbf455301784493b51959a559bb65d2 (127.24.83.2:37379)
W20260806 12:57:00.839139 32358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:00.839335 32358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:00.839358 32358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:00.840567 32358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:00.840610 32358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:00.840654 32358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:57:00.842079 32358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab
--principal=oryx/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:40855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=45203
--tserver_master_addrs=127.24.83.62:35127,127.24.83.61:40135,127.24.83.60:42009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:00.842316 32358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:00.842540 32358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:00.847872 32366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:00.847867 32364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:00.847868 32363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:00.848016 32358 server_base.cc:1034] running on GCE node
I20260806 12:57:00.848199 32358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:00.848392 32358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:00.849535 32358 hybrid_clock.cc:648] HybridClock initialized: now 1786021020849521 us; error 27 us; skew 500 ppm
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:00.852072 32358 init.cc:377] Logged in from keytab as oryx/127.24.83.3@KRBTEST.COM (short username oryx)
I20260806 12:57:00.852478 32358 webserver.cc:460] Webserver started at http://127.24.83.3:45203/ using document root <none> and password file <none>
I20260806 12:57:00.852686 32358 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:00.852762 32358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:00.854094 32358 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:00.854753 32373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.854945 32358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 12:57:00.855038 32358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f"
format_stamp: "Formatted at 2026-08-06 12:56:59 on dist-test-slave-zkpd"
server_key: "cf29cfa522fab06a091bdfc8ae6564a9"
server_key_iv: "5ab9abe6fcb8e155fed8752bfc85b5b8"
server_key_version: "encryptionkey@0"
I20260806 12:57:00.855432 32358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:00.873042 32358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:00.875541 32358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:00.875695 32358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:00.875973 32358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:00.876312 32358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:00.876358 32358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.876387 32358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:00.876405 32358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:00.877097 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.873782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42009 (local address 127.0.0.1:34078)
0806 12:57:00.873839 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.873853 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.873901 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.874140 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.874141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.874144 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:00.874197 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.874199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.875132 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.875133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.876085 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.876089 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.876905 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.876906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.876960 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.876964 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.877001 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.877043 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 12:57:00.877157 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.873879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34078 (local address 127.24.83.60:42009)
0806 12:57:00.873962 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.873964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.873975 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:57:00.874008 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.874009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.874034 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.874127 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:00.874274 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.875015 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.876170 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.876797 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.876888 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.876968 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.877028 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.877074 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 12:57:00.877429 32031 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 12:57:00.877588 32061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70b5b5dd010144199787f551a70ea8a6. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.877717 32061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: This master's current role is: LEARNER
I20260806 12:57:00.883119 32358 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:40855
I20260806 12:57:00.883211 32492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:40855 every 8 connection(s)
I20260806 12:57:00.883529 32358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:57:00.890202 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 32358
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.61@KRBTEST.COM
Aug 06 12:57:00 dist-test-slave-zkpd krb5kdc[31153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021020, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.60@KRBTEST.COM
I20260806 12:57:00.894979 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.884964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:57165 (local address 127.24.83.61:40135)
0806 12:57:00.885011 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.885013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.885205 (+ 192us) server_negotiation.cc:407] Connection header received
0806 12:57:00.885732 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.885734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.885745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.885798 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.886314 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.886799 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.888083 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.888360 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.888412 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.891465 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.891467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.891467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.891474 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.893328 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.893838 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.893839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.893840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.893914 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.894196 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.894197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.894197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.894323 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.894361 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.894703 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.894838 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 12:57:00.895750 32498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.884539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42009 (local address 127.24.83.3:37459)
0806 12:57:00.884755 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.884772 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.885409 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.885687 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.885688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.886020 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.886200 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.886205 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.887091 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.887092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.887991 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.887993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.888822 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.888823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.888899 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.889284 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.889289 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.892219 (+ 2930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.894426 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.894427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.894427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.894662 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.894908 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.894909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.894910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.894974 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.895263 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.895264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.895316 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.895376 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.895554 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":26,"thread_start_us":121,"threads_started":1}
I20260806 12:57:00.896273 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.884547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:37459 (local address 127.24.83.60:42009)
0806 12:57:00.885038 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.885040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.885049 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:57:00.885622 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.885622 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.885635 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.885681 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.886299 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.887011 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.888458 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.888750 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.888819 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.892313 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.892314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.892314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.892322 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.894245 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.894757 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.894758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.894758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.894830 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.895044 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.895045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.895045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.895180 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.895237 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.896115 (+ 878us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.896190 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469}
I20260806 12:57:00.896587 32494 heartbeater.cc:347] Connected to a master server at 127.24.83.60:42009
I20260806 12:57:00.896682 32494 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:00.896867 32494 heartbeater.cc:511] Master 127.24.83.60:42009 requested a full tablet report, sending...
I20260806 12:57:00.898289 32020 ts_manager.cc:194] Registered new tserver with Master: a45e4d5fb78f4ae4b94b9dbcbde1a08f (127.24.83.3:40855)
I20260806 12:57:00.898912 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.884898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:40135 (local address 127.24.83.3:57165)
0806 12:57:00.885012 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.885022 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.885423 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.885819 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.885820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.886104 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.886202 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.886208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.886906 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.886907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.887998 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.888000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.888440 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.888441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.888507 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.889015 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.889021 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.891370 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.893421 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.893428 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.893431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.893744 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.893988 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.893989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.893989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.894119 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.894406 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.894411 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.894554 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.895375 (+ 821us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.898780 (+ 3405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":3257,"thread_start_us":71,"threads_started":1}
I20260806 12:57:00.899295 32495 heartbeater.cc:347] Connected to a master server at 127.24.83.61:40135
I20260806 12:57:00.899355 32495 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:00.899441 32495 heartbeater.cc:511] Master 127.24.83.61:40135 requested a full tablet report, sending...
I20260806 12:57:00.899847 31821 ts_manager.cc:194] Registered new tserver with Master: a45e4d5fb78f4ae4b94b9dbcbde1a08f (127.24.83.3:40855)
I20260806 12:57:00.901935 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.884513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:55621 (local address 127.24.83.62:35127)
0806 12:57:00.884557 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.884559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.884836 (+ 277us) server_negotiation.cc:407] Connection header received
0806 12:57:00.885508 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.885510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.885522 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.885574 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.886364 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.887079 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.887878 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.888151 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.888214 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.893464 (+ 5250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.893475 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.893476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.893484 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.898662 (+ 5178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.900807 (+ 2145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.900808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.900808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.900869 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.901107 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.901108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.901108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.901239 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.901282 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.901502 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.901827 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:57:00.902185 32497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.884539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.3:55621)
0806 12:57:00.884730 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.884768 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.885423 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.885647 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.885650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.886210 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:00.886263 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.886265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.887143 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.887144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.887787 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.887790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.888225 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.888226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.888315 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.888832 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:00.888841 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:00.893052 (+ 4211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:00.900464 (+ 7412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.900465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.900465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.900713 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.900962 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:00.900963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:00.900963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:00.901030 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:00.901333 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:00.901334 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:00.901380 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.901546 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.901654 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":150,"threads_started":1}
I20260806 12:57:00.902776 32493 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35127
I20260806 12:57:00.902809 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.892085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38252 (local address 127.24.83.61:40135)
0806 12:57:00.892962 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.892964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.892971 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:57:00.893280 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.893281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.893330 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.894364 (+ 1034us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.895184 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.895917 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.897002 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.897361 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.897431 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.898990 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.898992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.898992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.898999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.901229 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.901621 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.901623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.901623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.901678 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.901928 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.901929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.901930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.902060 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.902176 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.902547 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.902708 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":858,"thread_start_us":56,"threads_started":1}
I20260806 12:57:00.902830 32493 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:00.902910 32493 heartbeater.cc:511] Master 127.24.83.62:35127 requested a full tablet report, sending...
I20260806 12:57:00.903149 31199 ts_manager.cc:194] Re-registered known tserver with Master: a45e4d5fb78f4ae4b94b9dbcbde1a08f (127.24.83.3:40855)
I20260806 12:57:00.903635 31199 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:55621
I20260806 12:57:00.904872 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.891944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41788 (local address 127.24.83.62:35127)
0806 12:57:00.892657 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.892660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.892668 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:00.892693 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.892693 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.892706 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.893820 (+ 1114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.893857 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.894599 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.896864 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.897143 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.897228 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.898943 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.898944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.898944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.899144 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.902804 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.904082 (+ 1278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.904083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.904084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.904150 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.904357 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.904358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.904358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.904465 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.904565 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.904671 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.904750 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":97,"threads_started":1}
I20260806 12:57:00.904922 32508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.892304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.24.83.60:42009)
0806 12:57:00.892403 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.892405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.894955 (+ 2550us) server_negotiation.cc:407] Connection header received
0806 12:57:00.894979 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.894980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.894992 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.895171 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:00.897279 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.898015 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.899044 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.899642 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.899702 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.901233 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:00.901235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:00.901235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:00.901243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:00.903550 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.904055 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.904056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.904057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.904113 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:00.904321 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:00.904322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:00.904323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:00.904460 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:00.904551 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.904688 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.904766 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":59,"threads_started":1}
I20260806 12:57:00.906572 31199 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:57:00.908222 31199 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:57:00.909108 31780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: NON_VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: true } } }
I20260806 12:57:00.909222 31780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:00.910059 32511 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6: attempting to promote NON_VOTER 63f73a41780e4913878e30fbdea6bc54 to VOTER
I20260806 12:57:00.910193 32511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } }
I20260806 12:57:00.910607 31831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Refusing update from remote peer 70b5b5dd010144199787f551a70ea8a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 12:57:00.910688 32031 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 LEARNER]: Refusing update from remote peer 70b5b5dd010144199787f551a70ea8a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 12:57:00.910801 31780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 12:57:00.910898 31781 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 12:57:00.911734 31780 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 63f73a41780e4913878e30fbdea6bc54 (127.24.83.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:00.912014 32031 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 63f73a41780e4913878e30fbdea6bc54 (127.24.83.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:00.912058 31831 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 63f73a41780e4913878e30fbdea6bc54 (127.24.83.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:00.912664 31780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:00.912747 31780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:00.912679 32511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:00.912848 32511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:00.912704 31781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:00.912891 31781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:00.913057 31983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:00.913146 31983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:57:00.913518 32061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70b5b5dd010144199787f551a70ea8a6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:00.913614 32061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:57:00.918953 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.915200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:37175 (local address 127.0.0.1:56968)
0806 12:57:00.915447 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.915458 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.915517 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.916159 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.916160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.916162 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:00.916209 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.916211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.917168 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.917169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.918031 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.918034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.918692 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.918693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.918752 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.918755 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.918851 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.918892 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231}
I20260806 12:57:00.919220 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.915523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56968 (local address 127.24.83.1:37175)
0806 12:57:00.915777 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.915782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.915793 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:57:00.915872 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.915874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.915924 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.916296 (+ 372us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:00.916358 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.917086 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.918130 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.918602 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.918680 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.918850 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.918879 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.918918 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":167,"threads_started":1}
I20260806 12:57:00.919955 32138 tablet_service.cc:1467] Processing CreateTablet for tablet e383ba87574a4f1ea498d6ae5fc4e19d (DEFAULT_TABLE table=test-table [id=5a4a93e2bf2249828a9f5be6754894d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:00.920104 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.915238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:40855 (local address 127.0.0.1:54444)
0806 12:57:00.915723 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.915733 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.915943 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.916351 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.916353 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.916355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:00.916411 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.916413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.917741 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.917741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.918557 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.918560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.919937 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.919938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.920001 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.920005 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.920028 (+ 23us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.920055 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":28,"thread_start_us":149,"threads_started":1}
I20260806 12:57:00.920324 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e383ba87574a4f1ea498d6ae5fc4e19d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:00.920596 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.915490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.24.83.3:40855)
0806 12:57:00.915966 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.915971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.915983 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:57:00.916055 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.916057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.916073 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.916275 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:00.916486 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.917651 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.918655 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.919810 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.920277 (+ 467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.920455 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.920486 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.920522 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":125,"threads_started":1}
I20260806 12:57:00.921263 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.915294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:37379 (local address 127.0.0.1:48758)
0806 12:57:00.916145 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:00.916155 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:00.916208 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:00.916415 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:00.916416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:00.916418 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:00.916454 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.916455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.917736 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.917737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.919816 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:00.919820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.920990 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.920992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:00.921064 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.921068 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:57:00.921152 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 12:57:00.921190 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":3,"thread_start_us":115,"threads_started":1}
I20260806 12:57:00.921348 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:00.915392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.24.83.2:37379)
0806 12:57:00.916158 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:00.916169 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:00.916210 (+ 41us) server_negotiation.cc:407] Connection header received
0806 12:57:00.916310 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:00.916312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:00.916329 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:00.916409 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:00.916513 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.917635 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.919922 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:00.920696 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:00.920770 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:00.920924 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:00.921180 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 12:57:00.921225 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":114,"threads_started":1}
I20260806 12:57:00.921305 32424 tablet_service.cc:1467] Processing CreateTablet for tablet e383ba87574a4f1ea498d6ae5fc4e19d (DEFAULT_TABLE table=test-table [id=5a4a93e2bf2249828a9f5be6754894d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:00.921705 32424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e383ba87574a4f1ea498d6ae5fc4e19d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:00.922021 32282 tablet_service.cc:1467] Processing CreateTablet for tablet e383ba87574a4f1ea498d6ae5fc4e19d (DEFAULT_TABLE table=test-table [id=5a4a93e2bf2249828a9f5be6754894d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:00.922369 32282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e383ba87574a4f1ea498d6ae5fc4e19d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:00.926060 32525 tablet_bootstrap.cc:492] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Bootstrap starting.
I20260806 12:57:00.926756 32526 tablet_bootstrap.cc:492] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: Bootstrap starting.
I20260806 12:57:00.927488 32525 tablet_bootstrap.cc:654] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:00.927735 32526 tablet_bootstrap.cc:654] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:00.928162 32525 log.cc:824] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:00.928153 32523 tablet_bootstrap.cc:492] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: Bootstrap starting.
I20260806 12:57:00.928627 32526 log.cc:824] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:00.929390 32525 tablet_bootstrap.cc:492] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: No bootstrap required, opened a new log
I20260806 12:57:00.929471 32525 ts_tablet_manager.cc:1397] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20260806 12:57:00.929659 32523 tablet_bootstrap.cc:654] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:00.929749 32526 tablet_bootstrap.cc:492] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: No bootstrap required, opened a new log
I20260806 12:57:00.929836 32526 ts_tablet_manager.cc:1397] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 12:57:00.930598 32523 log.cc:824] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:00.932354 32525 raft_consensus.cc:348] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:00.932538 32525 raft_consensus.cc:374] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:00.932582 32525 raft_consensus.cc:729] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45e4d5fb78f4ae4b94b9dbcbde1a08f, State: Initialized, Role: FOLLOWER
I20260806 12:57:00.932747 32525 consensus_queue.cc:260] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:00.932849 32523 tablet_bootstrap.cc:492] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: No bootstrap required, opened a new log
I20260806 12:57:00.932924 32523 ts_tablet_manager.cc:1397] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260806 12:57:00.933135 32525 ts_tablet_manager.cc:1428] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 12:57:00.933415 32493 heartbeater.cc:503] Master 127.24.83.62:35127 was elected leader, sending a full tablet report...
I20260806 12:57:00.933389 32526 raft_consensus.cc:348] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:00.933542 32526 raft_consensus.cc:374] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:00.933574 32526 raft_consensus.cc:729] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdbf455301784493b51959a559bb65d2, State: Initialized, Role: FOLLOWER
I20260806 12:57:00.933727 32526 consensus_queue.cc:260] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:00.934058 32526 ts_tablet_manager.cc:1428] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260806 12:57:00.934235 32351 heartbeater.cc:503] Master 127.24.83.62:35127 was elected leader, sending a full tablet report...
I20260806 12:57:00.936419 32523 raft_consensus.cc:348] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:00.936540 32523 raft_consensus.cc:374] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:00.936561 32523 raft_consensus.cc:729] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30d865941464d91bd03c5cc1cb2bee2, State: Initialized, Role: FOLLOWER
I20260806 12:57:00.936781 32523 consensus_queue.cc:260] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:00.937165 32523 ts_tablet_manager.cc:1428] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 12:57:00.937290 32208 heartbeater.cc:503] Master 127.24.83.62:35127 was elected leader, sending a full tablet report...
W20260806 12:57:00.988646 32354 tablet.cc:2369] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:57:01.087550 32211 tablet.cc:2369] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:57:01.134650 32496 tablet.cc:2369] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:57:01.298231 32532 raft_consensus.cc:482] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:57:01.298352 32532 raft_consensus.cc:504] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:01.298821 32532 leader_election.cc:288] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45e4d5fb78f4ae4b94b9dbcbde1a08f (127.24.83.3:40855), bdbf455301784493b51959a559bb65d2 (127.24.83.2:37379)
I20260806 12:57:01.301756 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:01.299091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:48751 (local address 127.24.83.2:37379)
0806 12:57:01.299136 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:01.299138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:01.299251 (+ 113us) server_negotiation.cc:407] Connection header received
0806 12:57:01.299323 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:01.299323 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:01.299333 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:01.299409 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:01.299585 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.300090 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.301044 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.301398 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.301450 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:01.301459 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:01.301644 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 12:57:01.301677 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 12:57:01.301781 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:01.299030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:37379 (local address 127.24.83.1:48751)
0806 12:57:01.299160 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:01.299170 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:01.299253 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:01.299422 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:01.299423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:01.299425 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:01.299516 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:01.299518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.300180 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.300181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:01.300959 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:01.300961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.301490 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.301491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:01.301567 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:01.301572 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:57:01.301630 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 12:57:01.301675 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260806 12:57:01.301892 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:01.299015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:40855 (local address 127.24.83.1:54927)
0806 12:57:01.299142 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:01.299153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:01.299216 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:01.299378 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:01.299379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:01.299382 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:01.299444 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:01.299446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.300133 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.300134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:01.300964 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:01.300967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.301649 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.301650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:01.301704 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:01.301708 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:57:01.301763 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 12:57:01.301804 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":35,"thread_start_us":60,"threads_started":1}
I20260806 12:57:01.301916 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:01.299078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:54927 (local address 127.24.83.3:40855)
0806 12:57:01.299125 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:01.299127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:01.299225 (+ 98us) server_negotiation.cc:407] Connection header received
0806 12:57:01.299296 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:01.299297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:01.299312 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:01.299364 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:01.299515 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.300063 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.301076 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.301563 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.301639 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:01.301648 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:01.301782 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 12:57:01.301834 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:57:01.302440 32303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e383ba87574a4f1ea498d6ae5fc4e19d" candidate_uuid: "a30d865941464d91bd03c5cc1cb2bee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdbf455301784493b51959a559bb65d2" is_pre_election: true
I20260806 12:57:01.302469 32446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e383ba87574a4f1ea498d6ae5fc4e19d" candidate_uuid: "a30d865941464d91bd03c5cc1cb2bee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" is_pre_election: true
I20260806 12:57:01.302623 32303 raft_consensus.cc:2454] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a30d865941464d91bd03c5cc1cb2bee2 in term 0.
I20260806 12:57:01.302624 32446 raft_consensus.cc:2454] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a30d865941464d91bd03c5cc1cb2bee2 in term 0.
I20260806 12:57:01.302843 32091 leader_election.cc:302] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30d865941464d91bd03c5cc1cb2bee2, a45e4d5fb78f4ae4b94b9dbcbde1a08f; no voters:
I20260806 12:57:01.302973 32532 raft_consensus.cc:2781] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:57:01.303028 32532 raft_consensus.cc:482] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:57:01.303054 32532 raft_consensus.cc:3037] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:01.303857 32532 raft_consensus.cc:504] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:01.304034 32532 leader_election.cc:288] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [CANDIDATE]: Term 1 election: Requested vote from peers a45e4d5fb78f4ae4b94b9dbcbde1a08f (127.24.83.3:40855), bdbf455301784493b51959a559bb65d2 (127.24.83.2:37379)
I20260806 12:57:01.304170 32446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e383ba87574a4f1ea498d6ae5fc4e19d" candidate_uuid: "a30d865941464d91bd03c5cc1cb2bee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f"
I20260806 12:57:01.304214 32303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e383ba87574a4f1ea498d6ae5fc4e19d" candidate_uuid: "a30d865941464d91bd03c5cc1cb2bee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdbf455301784493b51959a559bb65d2"
I20260806 12:57:01.304276 32303 raft_consensus.cc:3037] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:01.304270 32446 raft_consensus.cc:3037] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:01.305197 32303 raft_consensus.cc:2454] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30d865941464d91bd03c5cc1cb2bee2 in term 1.
I20260806 12:57:01.305388 32089 leader_election.cc:302] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30d865941464d91bd03c5cc1cb2bee2, bdbf455301784493b51959a559bb65d2; no voters:
I20260806 12:57:01.305401 32446 raft_consensus.cc:2454] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30d865941464d91bd03c5cc1cb2bee2 in term 1.
I20260806 12:57:01.305581 32532 raft_consensus.cc:2781] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:01.305737 32532 raft_consensus.cc:686] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 1 LEADER]: Becoming Leader. State: Replica: a30d865941464d91bd03c5cc1cb2bee2, State: Running, Role: LEADER
I20260806 12:57:01.305860 32532 consensus_queue.cc:237] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:01.306681 31199 catalog_manager.cc:5495] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a30d865941464d91bd03c5cc1cb2bee2 (127.24.83.1). New cstate: current_term: 1 leader_uuid: "a30d865941464d91bd03c5cc1cb2bee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } health_report { overall_health: UNKNOWN } } }
I20260806 12:57:01.363651 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:01.361420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56974 (local address 127.24.83.1:37175)
0806 12:57:01.361478 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:01.361479 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:01.361487 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:01.361538 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:01.361539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:01.361559 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:01.361615 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:57:01.361741 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.362221 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.362970 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.363158 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.363208 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:01.363334 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:57:01.363433 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:57:01.363493 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:01.363524 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 12:57:01.363540 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 12:57:01.365778 32303 raft_consensus.cc:1261] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2 [term 1 FOLLOWER]: Refusing update from remote peer a30d865941464d91bd03c5cc1cb2bee2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:01.365778 32446 raft_consensus.cc:1261] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 1 FOLLOWER]: Refusing update from remote peer a30d865941464d91bd03c5cc1cb2bee2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:01.366129 32535 consensus_queue.cc:1035] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:01.366248 32532 consensus_queue.cc:1035] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:01.371827 31199 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:01.371960 31199 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:01.373553 31199 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70b5b5dd010144199787f551a70ea8a6: Sending DeleteTablet for 3 replicas of tablet e383ba87574a4f1ea498d6ae5fc4e19d
I20260806 12:57:01.373947 32282 tablet_service.cc:1514] Processing DeleteTablet for tablet e383ba87574a4f1ea498d6ae5fc4e19d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:01 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:48758
I20260806 12:57:01.374046 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet e383ba87574a4f1ea498d6ae5fc4e19d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:01 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56968
I20260806 12:57:01.374104 32424 tablet_service.cc:1514] Processing DeleteTablet for tablet e383ba87574a4f1ea498d6ae5fc4e19d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:01 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:54444
I20260806 12:57:01.374166 32548 tablet_replica.cc:330] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: stopping tablet replica
I20260806 12:57:01.374253 32548 raft_consensus.cc:2229] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:01.374356 32548 raft_consensus.cc:2258] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:01.374936 32548 ts_tablet_manager.cc:1905] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:01.375427 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 31167
I20260806 12:57:01.376932 32548 ts_tablet_manager.cc:1918] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:01.377018 32548 log.cc:1196] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/e383ba87574a4f1ea498d6ae5fc4e19d
I20260806 12:57:01.377274 32548 ts_tablet_manager.cc:1939] T e383ba87574a4f1ea498d6ae5fc4e19d P bdbf455301784493b51959a559bb65d2: Deleting consensus metadata
I20260806 12:57:01.380021 32550 tablet_replica.cc:330] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: stopping tablet replica
I20260806 12:57:01.380131 32550 raft_consensus.cc:2229] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:01.380240 32550 raft_consensus.cc:2258] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:01.380398 32549 tablet_replica.cc:330] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: stopping tablet replica
I20260806 12:57:01.380496 32549 raft_consensus.cc:2229] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:57:01.380586 32549 raft_consensus.cc:2258] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:01.380980 32549 ts_tablet_manager.cc:1905] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:01.382800 32550 ts_tablet_manager.cc:1905] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 12:57:01.383836 32234 connection.cc:535] server connection from 127.0.0.1:48758 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 12:57:01.384567 32550 ts_tablet_manager.cc:1918] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:01.384649 32550 log.cc:1196] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/e383ba87574a4f1ea498d6ae5fc4e19d
I20260806 12:57:01.384915 32550 ts_tablet_manager.cc:1939] T e383ba87574a4f1ea498d6ae5fc4e19d P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Deleting consensus metadata
I20260806 12:57:01.385097 32549 ts_tablet_manager.cc:1918] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:01.385159 32549 log.cc:1196] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-0/wal/wals/e383ba87574a4f1ea498d6ae5fc4e19d
I20260806 12:57:01.385404 32549 ts_tablet_manager.cc:1939] T e383ba87574a4f1ea498d6ae5fc4e19d P a30d865941464d91bd03c5cc1cb2bee2: Deleting consensus metadata
W20260806 12:57:01.385713 32088 connection.cc:441] server connection from 127.0.0.1:56968 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56968 (request call id 1) could send its response
W20260806 12:57:01.386062 32376 connection.cc:441] server connection from 127.0.0.1:54444 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54444 (request call id 1) could send its response
I20260806 12:57:01.386806 32508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:01.374944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34092 (local address 127.24.83.60:42009)
0806 12:57:01.374986 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:01.374988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:01.374994 (+ 6us) server_negotiation.cc:407] Connection header received
0806 12:57:01.375088 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:01.375089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:01.375103 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:01.375132 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:01.378144 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.378927 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.380524 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.380882 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.380931 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:01.383047 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:01.383049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:01.383049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:01.383057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:01.385067 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:01.385687 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:01.385688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:01.385688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:01.385749 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:01.386044 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:01.386045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:01.386045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:01.386187 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:01.386266 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:01.386575 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 12:57:01.386673 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:57:01.393435 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:01.386051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.24.83.60:42009)
0806 12:57:01.386093 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:01.386095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:01.386344 (+ 249us) server_negotiation.cc:407] Connection header received
0806 12:57:01.386448 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:01.386448 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:01.386458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:01.386487 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:01.387177 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.387930 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.388979 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.389272 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.389340 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:01.390714 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:01.390716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:01.390717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:01.390724 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:01.392254 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:01.392596 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:01.392597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:01.392598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:01.392655 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:01.392864 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:01.392865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:01.392866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:01.393010 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:01.393115 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:01.393262 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 12:57:01.393341 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 12:57:01.394737 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:01.386606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38260 (local address 127.24.83.61:40135)
0806 12:57:01.386653 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:01.386655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:01.386983 (+ 328us) server_negotiation.cc:407] Connection header received
0806 12:57:01.387071 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:01.387072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:01.387087 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:01.387117 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:01.388140 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.388615 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.389807 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:01.389997 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:01.390101 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:01.391511 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:01.391513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:01.391513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:01.391521 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:01.393611 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:01.394059 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:01.394060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:01.394060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:01.394115 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:01.394314 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:01.394314 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:01.394315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:01.394415 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:01.394485 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:01.394613 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 12:57:01.394678 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:57:02.370940 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.370565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.3:43449)
0806 12:57:02.370768 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.370851 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":78,"threads_started":1}
W20260806 12:57:02.371213 32493 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:35127 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
I20260806 12:57:02.371814 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.371570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.1:35341)
0806 12:57:02.371691 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.371770 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
W20260806 12:57:02.372117 32208 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:35127 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
I20260806 12:57:02.374315 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.374119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.2:53285)
0806 12:57:02.374219 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.374280 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":60,"threads_started":1}
W20260806 12:57:02.374495 32351 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:35127 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
I20260806 12:57:02.873690 32562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70b5b5dd010144199787f551a70ea8a6)
I20260806 12:57:02.873908 32563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70b5b5dd010144199787f551a70ea8a6)
I20260806 12:57:02.873916 32562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } }
I20260806 12:57:02.874012 32563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } }
I20260806 12:57:02.874199 32563 leader_election.cc:288] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127), 7ae452e605104e909ae0c5ca954c82ba (127.24.83.61:40135)
I20260806 12:57:02.874253 32562 leader_election.cc:288] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127), 63f73a41780e4913878e30fbdea6bc54 (127.24.83.60:42009)
I20260806 12:57:02.874667 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.874308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42262)
0806 12:57:02.874425 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.874617 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":54,"threads_started":1}
I20260806 12:57:02.874754 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.874304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42256)
0806 12:57:02.874660 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.874710 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":44,"threads_started":1}
W20260806 12:57:02.874782 31805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
W20260806 12:57:02.874900 32004 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
I20260806 12:57:02.875524 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.875420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42288)
0806 12:57:02.875470 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.875500 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 12:57:02.875595 32004 leader_election.cc:334] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127): Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
I20260806 12:57:02.875546 32570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.875317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42274)
0806 12:57:02.875475 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.875516 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260806 12:57:02.875723 31805 leader_election.cc:334] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127): Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
I20260806 12:57:02.877750 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.874455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:40135 (local address 127.0.0.1:38268)
0806 12:57:02.874933 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.875004 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:02.875060 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:02.875227 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:02.875228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:02.875231 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:02.875291 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:02.875294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:02.876217 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:02.876218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:02.876961 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:02.876963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:02.877508 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:02.877509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:02.877568 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:02.877571 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:57:02.877628 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 12:57:02.877671 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":37,"threads_started":1}
I20260806 12:57:02.877789 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.874506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38268 (local address 127.24.83.61:40135)
0806 12:57:02.875039 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:02.875041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:02.875049 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:02.875120 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:02.875121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:02.875132 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:02.875210 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:02.875368 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:02.876138 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:02.877048 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:02.877421 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:02.877496 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:02.877508 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:02.877646 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 12:57:02.877686 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":57,"threads_started":1}
I20260806 12:57:02.877847 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.874648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42009 (local address 127.0.0.1:34104)
0806 12:57:02.874963 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.874974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:02.875027 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:02.875375 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:02.875376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:02.875378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:02.875425 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:02.875427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:02.876260 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:02.876261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:02.876994 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:02.876995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:02.877531 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:02.877531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:02.877592 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:02.877596 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:57:02.877670 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 12:57:02.877710 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":50,"threads_started":1}
I20260806 12:57:02.877837 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.874572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34104 (local address 127.24.83.60:42009)
0806 12:57:02.875115 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:02.875117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:02.875123 (+ 6us) server_negotiation.cc:407] Connection header received
0806 12:57:02.875145 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:02.875145 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:02.875157 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:02.875494 (+ 337us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:02.875531 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:02.876180 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:02.877092 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:02.877454 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:02.877530 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:02.877541 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:02.877678 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 12:57:02.877717 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":43,"threads_started":1}
I20260806 12:57:02.878136 32031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ae452e605104e909ae0c5ca954c82ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "63f73a41780e4913878e30fbdea6bc54" is_pre_election: true
I20260806 12:57:02.878237 32031 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ae452e605104e909ae0c5ca954c82ba in term 1.
I20260806 12:57:02.878201 31831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63f73a41780e4913878e30fbdea6bc54" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7ae452e605104e909ae0c5ca954c82ba" is_pre_election: true
I20260806 12:57:02.878314 31831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63f73a41780e4913878e30fbdea6bc54 in term 1.
I20260806 12:57:02.878453 31808 leader_election.cc:302] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 63f73a41780e4913878e30fbdea6bc54, 7ae452e605104e909ae0c5ca954c82ba; no voters: 70b5b5dd010144199787f551a70ea8a6
I20260806 12:57:02.878468 32004 leader_election.cc:302] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 63f73a41780e4913878e30fbdea6bc54, 7ae452e605104e909ae0c5ca954c82ba; no voters: 70b5b5dd010144199787f551a70ea8a6
I20260806 12:57:02.878561 32563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 12:57:02.878567 32562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 12:57:02.878607 32562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Starting leader election (detected failure of leader 70b5b5dd010144199787f551a70ea8a6)
I20260806 12:57:02.878607 32563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 70b5b5dd010144199787f551a70ea8a6)
I20260806 12:57:02.878630 32563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:57:02.878631 32562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:57:02.879297 32563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } }
I20260806 12:57:02.879307 32562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } }
I20260806 12:57:02.879422 32563 leader_election.cc:288] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 2 election: Requested vote from peers 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127), 7ae452e605104e909ae0c5ca954c82ba (127.24.83.61:40135)
I20260806 12:57:02.879447 32562 leader_election.cc:288] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [CANDIDATE]: Term 2 election: Requested vote from peers 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127), 63f73a41780e4913878e30fbdea6bc54 (127.24.83.60:42009)
I20260806 12:57:02.879602 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.879506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42296)
0806 12:57:02.879545 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.879577 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 12:57:02.879618 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.879515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42304)
0806 12:57:02.879576 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.879595 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 12:57:02.879642 31831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63f73a41780e4913878e30fbdea6bc54" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7ae452e605104e909ae0c5ca954c82ba"
I20260806 12:57:02.879726 31831 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 63f73a41780e4913878e30fbdea6bc54 in current term 2: Already voted for candidate 7ae452e605104e909ae0c5ca954c82ba in this term.
I20260806 12:57:02.879736 32031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ae452e605104e909ae0c5ca954c82ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "63f73a41780e4913878e30fbdea6bc54"
I20260806 12:57:02.879812 32031 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7ae452e605104e909ae0c5ca954c82ba in current term 2: Already voted for candidate 63f73a41780e4913878e30fbdea6bc54 in this term.
I20260806 12:57:02.880229 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.880136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42308)
0806 12:57:02.880189 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.880211 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 12:57:02.880237 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:02.880148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42322)
0806 12:57:02.880191 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:02.880214 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 12:57:02.880306 32004 leader_election.cc:334] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127): Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
W20260806 12:57:02.880308 31805 leader_election.cc:334] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127): Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
I20260806 12:57:02.880344 31805 leader_election.cc:302] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ae452e605104e909ae0c5ca954c82ba; no voters: 63f73a41780e4913878e30fbdea6bc54, 70b5b5dd010144199787f551a70ea8a6
I20260806 12:57:02.880347 32004 leader_election.cc:302] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63f73a41780e4913878e30fbdea6bc54; no voters: 70b5b5dd010144199787f551a70ea8a6, 7ae452e605104e909ae0c5ca954c82ba
I20260806 12:57:02.880421 32563 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 12:57:02.880427 32562 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 12:57:03.371793 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:03.371513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.3:59845)
0806 12:57:03.371674 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:03.371730 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260806 12:57:03.372720 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:03.372525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.1:45773)
0806 12:57:03.372644 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:03.372678 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260806 12:57:03.374912 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:03.374739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.2:57017)
0806 12:57:03.374842 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:03.374877 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260806 12:57:04.372630 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.372298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.3:59287)
0806 12:57:04.372466 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.372561 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":111,"threads_started":1}
I20260806 12:57:04.373405 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.373150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.1:45097)
0806 12:57:04.373297 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.373354 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260806 12:57:04.375432 32578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.375262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.2:46639)
0806 12:57:04.375359 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.375395 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":46,"threads_started":1}
I20260806 12:57:04.471253 32579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:57:04.471350 32579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } }
I20260806 12:57:04.471530 32579 leader_election.cc:288] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127), 7ae452e605104e909ae0c5ca954c82ba (127.24.83.61:40135)
I20260806 12:57:04.471935 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.471680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42542)
0806 12:57:04.471796 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.471849 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
I20260806 12:57:04.471998 31831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63f73a41780e4913878e30fbdea6bc54" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7ae452e605104e909ae0c5ca954c82ba" is_pre_election: true
I20260806 12:57:04.472115 31831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63f73a41780e4913878e30fbdea6bc54 in term 2.
I20260806 12:57:04.472357 32004 leader_election.cc:302] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63f73a41780e4913878e30fbdea6bc54, 7ae452e605104e909ae0c5ca954c82ba; no voters:
I20260806 12:57:04.472424 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.472294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42544)
0806 12:57:04.472339 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.472369 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 12:57:04.472466 32004 leader_election.cc:334] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127): Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
I20260806 12:57:04.472518 32579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 12:57:04.472556 32579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:57:04.472575 32579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 2 FOLLOWER]: Advancing to term 3
I20260806 12:57:04.473608 32579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } }
I20260806 12:57:04.473781 32579 leader_election.cc:288] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 3 election: Requested vote from peers 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127), 7ae452e605104e909ae0c5ca954c82ba (127.24.83.61:40135)
I20260806 12:57:04.473954 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.473868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42574)
0806 12:57:04.473901 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.473930 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 12:57:04.473999 31831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63f73a41780e4913878e30fbdea6bc54" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7ae452e605104e909ae0c5ca954c82ba"
I20260806 12:57:04.474081 31831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 2 FOLLOWER]: Advancing to term 3
I20260806 12:57:04.474210 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.474144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42576)
0806 12:57:04.474170 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.474191 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 12:57:04.474283 32004 leader_election.cc:334] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127): Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
I20260806 12:57:04.474768 31831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63f73a41780e4913878e30fbdea6bc54 in term 3.
I20260806 12:57:04.474952 32004 leader_election.cc:302] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 63f73a41780e4913878e30fbdea6bc54, 7ae452e605104e909ae0c5ca954c82ba; no voters: 70b5b5dd010144199787f551a70ea8a6
I20260806 12:57:04.475029 32579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 12:57:04.475123 32579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [term 3 LEADER]: Becoming Leader. State: Replica: 63f73a41780e4913878e30fbdea6bc54, State: Running, Role: LEADER
I20260806 12:57:04.475194 32579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } }
I20260806 12:57:04.475522 32583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63f73a41780e4913878e30fbdea6bc54. Latest consensus state: current_term: 3 leader_uuid: "63f73a41780e4913878e30fbdea6bc54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:04.475629 32583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:04.475813 32585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:57:04.476341 32585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:57:04.476408 32585 catalog_manager.cc:1259] Loaded cluster ID: c298d5eafbc745d6945d505298714ac0
I20260806 12:57:04.476432 32585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:57:04.476573 32585 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:57:04.476624 32585 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: Loaded TSK: 0
I20260806 12:57:04.476719 32585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:57:04.555792 32019 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:57:04.557505 32019 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:57:04.558528 31831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [term 3 FOLLOWER]: Refusing update from remote peer 63f73a41780e4913878e30fbdea6bc54: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 12:57:04.558549 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.558446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42592)
0806 12:57:04.558494 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.558524 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 12:57:04.558805 32579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 12:57:04.558979 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.558852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.0.0.1:42604)
0806 12:57:04.558922 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.558952 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 12:57:04.559039 32004 consensus_peers.cc:487] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 -> Peer 70b5b5dd010144199787f551a70ea8a6 (127.24.83.62:35127): Couldn't send request to peer 70b5b5dd010144199787f551a70ea8a6. Status: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 12:57:04.559448 32587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63f73a41780e4913878e30fbdea6bc54. Latest consensus state: current_term: 3 leader_uuid: "63f73a41780e4913878e30fbdea6bc54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:04.559554 32587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:57:04.559957 32579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "63f73a41780e4913878e30fbdea6bc54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:04.560072 32579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:04.560078 32587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "63f73a41780e4913878e30fbdea6bc54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "70b5b5dd010144199787f551a70ea8a6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35127 } } peers { permanent_uuid: "7ae452e605104e909ae0c5ca954c82ba" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 40135 } attrs { promote: false } } peers { permanent_uuid: "63f73a41780e4913878e30fbdea6bc54" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42009 } attrs { promote: false } } }
I20260806 12:57:04.560165 32587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae452e605104e909ae0c5ca954c82ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:57:04.564996 32593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.561827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:40855 (local address 127.0.0.1:54452)
0806 12:57:04.562031 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.562042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:04.562081 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:04.562316 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:04.562317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:04.562319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:04.562370 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:04.562372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.563287 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.563288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:04.564086 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:04.564089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.564817 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.564817 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:04.564870 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:04.564874 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:57:04.564919 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 12:57:04.564949 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260806 12:57:04.565049 32598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.562015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54452 (local address 127.24.83.3:40855)
0806 12:57:04.562116 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:04.562118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:04.562125 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:57:04.562151 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:04.562152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:04.562163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:04.562279 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:04.562452 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.563178 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.564178 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.564736 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.564801 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:04.564812 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:04.564940 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 12:57:04.564977 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":53,"threads_started":1}
I20260806 12:57:04.565555 32424 tablet_service.cc:1467] Processing CreateTablet for tablet 64723e16d378409695be5171e4a8b68e (DEFAULT_TABLE table=test-table [id=c2768637c1044ecbb77edb6db89c9912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:04.565690 32424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64723e16d378409695be5171e4a8b68e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:04.566160 32596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.561897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:37379 (local address 127.0.0.1:48774)
0806 12:57:04.562347 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.562356 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:04.562415 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:04.562569 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:04.562570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:04.562572 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:04.562606 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:04.562607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.563187 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.563188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:04.564341 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:04.564344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.565927 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.565927 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:04.565972 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:04.565975 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:57:04.566044 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 12:57:04.566080 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":5,"thread_start_us":79,"threads_started":1}
I20260806 12:57:04.566262 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.561866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56986 (local address 127.24.83.1:37175)
0806 12:57:04.562179 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:04.562182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:04.562190 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:04.562226 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:04.562227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:04.562236 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:04.562469 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:04.562512 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.563202 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.564420 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.565956 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.566017 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:04.566031 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:04.566147 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 12:57:04.566190 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":84,"threads_started":1}
I20260806 12:57:04.566416 32597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.561993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48774 (local address 127.24.83.2:37379)
0806 12:57:04.562087 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:04.562090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:04.562464 (+ 374us) server_negotiation.cc:407] Connection header received
0806 12:57:04.562495 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:04.562496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:04.562505 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:04.562570 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:04.562658 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.563112 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.565237 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.565846 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.565925 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:04.565938 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:04.566298 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 12:57:04.566350 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":43,"threads_started":1}
I20260806 12:57:04.566265 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.561807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:37175 (local address 127.0.0.1:56986)
0806 12:57:04.562009 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.562021 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:04.562076 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:04.562300 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:04.562301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:04.562304 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:04.562358 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:04.562360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.563270 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.563271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:04.564341 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:04.564344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.566021 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.566022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:04.566077 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:04.566080 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:57:04.566127 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 12:57:04.566161 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":24}
I20260806 12:57:04.566747 32282 tablet_service.cc:1467] Processing CreateTablet for tablet 64723e16d378409695be5171e4a8b68e (DEFAULT_TABLE table=test-table [id=c2768637c1044ecbb77edb6db89c9912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:04.566875 32282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64723e16d378409695be5171e4a8b68e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:04.567217 32599 tablet_bootstrap.cc:492] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Bootstrap starting.
I20260806 12:57:04.567266 32138 tablet_service.cc:1467] Processing CreateTablet for tablet 64723e16d378409695be5171e4a8b68e (DEFAULT_TABLE table=test-table [id=c2768637c1044ecbb77edb6db89c9912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:04.567373 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64723e16d378409695be5171e4a8b68e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:04.567615 32599 tablet_bootstrap.cc:654] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:04.568301 32599 tablet_bootstrap.cc:492] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f: No bootstrap required, opened a new log
I20260806 12:57:04.568367 32599 ts_tablet_manager.cc:1397] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 12:57:04.568327 32601 tablet_bootstrap.cc:492] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2: Bootstrap starting.
I20260806 12:57:04.568516 32599 raft_consensus.cc:348] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:04.568588 32599 raft_consensus.cc:374] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:04.568609 32599 raft_consensus.cc:729] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45e4d5fb78f4ae4b94b9dbcbde1a08f, State: Initialized, Role: FOLLOWER
I20260806 12:57:04.568676 32599 consensus_queue.cc:260] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:04.568930 32603 tablet_bootstrap.cc:492] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2: Bootstrap starting.
I20260806 12:57:04.568904 32599 ts_tablet_manager.cc:1428] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 12:57:04.569078 32601 tablet_bootstrap.cc:654] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:04.569208 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.569103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.3:41639)
0806 12:57:04.569145 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.569170 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 12:57:04.569506 32603 tablet_bootstrap.cc:654] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:04.569830 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.569548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.3:58603)
0806 12:57:04.569770 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.569799 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
I20260806 12:57:04.569928 32494 heartbeater.cc:503] Master 127.24.83.60:42009 was elected leader, sending a full tablet report...
I20260806 12:57:04.570293 32601 tablet_bootstrap.cc:492] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2: No bootstrap required, opened a new log
I20260806 12:57:04.570345 32601 ts_tablet_manager.cc:1397] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:57:04.570411 32603 tablet_bootstrap.cc:492] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2: No bootstrap required, opened a new log
I20260806 12:57:04.570482 32603 ts_tablet_manager.cc:1397] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:57:04.570494 32601 raft_consensus.cc:348] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:04.570564 32601 raft_consensus.cc:374] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:04.570593 32601 raft_consensus.cc:729] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdbf455301784493b51959a559bb65d2, State: Initialized, Role: FOLLOWER
I20260806 12:57:04.570595 32603 raft_consensus.cc:348] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:04.570667 32603 raft_consensus.cc:374] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:04.570644 32601 consensus_queue.cc:260] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:04.570698 32603 raft_consensus.cc:729] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30d865941464d91bd03c5cc1cb2bee2, State: Initialized, Role: FOLLOWER
I20260806 12:57:04.570748 32603 consensus_queue.cc:260] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:04.570832 32601 ts_tablet_manager.cc:1428] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:04.570920 32603 ts_tablet_manager.cc:1428] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:04.571151 32578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.570982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.2:42431)
0806 12:57:04.571090 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.571120 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 12:57:04.571436 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.571323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.1:53955)
0806 12:57:04.571380 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.571411 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 12:57:04.571491 32578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.571378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.2:46401)
0806 12:57:04.571437 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.571466 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 12:57:04.571774 32209 heartbeater.cc:503] Master 127.24.83.60:42009 was elected leader, sending a full tablet report...
I20260806 12:57:04.657387 32606 raft_consensus.cc:482] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:57:04.657517 32606 raft_consensus.cc:504] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:04.657953 32606 leader_election.cc:288] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30d865941464d91bd03c5cc1cb2bee2 (127.24.83.1:37175), a45e4d5fb78f4ae4b94b9dbcbde1a08f (127.24.83.3:40855)
I20260806 12:57:04.660854 32578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.658093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:37175 (local address 127.24.83.2:35523)
0806 12:57:04.658237 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.658248 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:04.658310 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:04.658450 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:04.658451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:04.658453 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:04.658532 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:04.658534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.659207 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.659208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:04.660038 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:04.660040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.660617 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.660618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:04.660688 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:04.660692 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:57:04.660743 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 12:57:04.660774 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 12:57:04.660914 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.658222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:35523 (local address 127.24.83.1:37175)
0806 12:57:04.658285 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:04.658296 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:04.658307 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:57:04.658378 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:04.658379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:04.658392 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:04.658445 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:04.658624 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.659125 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.660122 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.660534 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.660591 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:04.660601 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:04.660767 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 12:57:04.660847 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 12:57:04.661047 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.658110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:40855 (local address 127.24.83.2:44335)
0806 12:57:04.658309 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.658317 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:04.658402 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:04.658589 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:04.658590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:04.658592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:04.658672 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:04.658673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.659317 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.659318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:04.660136 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:04.660138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.660694 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.660695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:04.660771 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:04.660776 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:57:04.660840 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 12:57:04.660876 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":114,"threads_started":1}
I20260806 12:57:04.661454 32160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64723e16d378409695be5171e4a8b68e" candidate_uuid: "bdbf455301784493b51959a559bb65d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30d865941464d91bd03c5cc1cb2bee2" is_pre_election: true
I20260806 12:57:04.661628 32160 raft_consensus.cc:2454] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdbf455301784493b51959a559bb65d2 in term 0.
I20260806 12:57:04.661886 32598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.658183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:44335 (local address 127.24.83.3:40855)
0806 12:57:04.658223 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:04.658226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:04.658379 (+ 153us) server_negotiation.cc:407] Connection header received
0806 12:57:04.658498 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:04.658499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:04.658514 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:04.658585 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:04.658754 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.659253 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.660220 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.660587 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.660658 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:04.660669 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:04.661747 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0806 12:57:04.661807 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260806 12:57:04.661887 32235 leader_election.cc:302] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30d865941464d91bd03c5cc1cb2bee2, bdbf455301784493b51959a559bb65d2; no voters:
I20260806 12:57:04.662063 32606 raft_consensus.cc:2781] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:57:04.662104 32606 raft_consensus.cc:482] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:57:04.662134 32606 raft_consensus.cc:3037] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:04.662096 32446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64723e16d378409695be5171e4a8b68e" candidate_uuid: "bdbf455301784493b51959a559bb65d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" is_pre_election: true
I20260806 12:57:04.662186 32446 raft_consensus.cc:2454] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdbf455301784493b51959a559bb65d2 in term 0.
I20260806 12:57:04.662895 32606 raft_consensus.cc:504] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:04.663054 32606 leader_election.cc:288] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [CANDIDATE]: Term 1 election: Requested vote from peers a30d865941464d91bd03c5cc1cb2bee2 (127.24.83.1:37175), a45e4d5fb78f4ae4b94b9dbcbde1a08f (127.24.83.3:40855)
I20260806 12:57:04.663206 32160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64723e16d378409695be5171e4a8b68e" candidate_uuid: "bdbf455301784493b51959a559bb65d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30d865941464d91bd03c5cc1cb2bee2"
I20260806 12:57:04.663226 32446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64723e16d378409695be5171e4a8b68e" candidate_uuid: "bdbf455301784493b51959a559bb65d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f"
I20260806 12:57:04.663295 32160 raft_consensus.cc:3037] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:04.663319 32446 raft_consensus.cc:3037] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:04.663990 32446 raft_consensus.cc:2454] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdbf455301784493b51959a559bb65d2 in term 1.
I20260806 12:57:04.664021 32160 raft_consensus.cc:2454] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdbf455301784493b51959a559bb65d2 in term 1.
I20260806 12:57:04.664191 32237 leader_election.cc:302] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45e4d5fb78f4ae4b94b9dbcbde1a08f, bdbf455301784493b51959a559bb65d2; no voters:
I20260806 12:57:04.664296 32606 raft_consensus.cc:2781] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:04.664445 32606 raft_consensus.cc:686] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 1 LEADER]: Becoming Leader. State: Replica: bdbf455301784493b51959a559bb65d2, State: Running, Role: LEADER
I20260806 12:57:04.664551 32352 heartbeater.cc:503] Master 127.24.83.60:42009 was elected leader, sending a full tablet report...
I20260806 12:57:04.664501 32606 consensus_queue.cc:237] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } }
I20260806 12:57:04.664793 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.664706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.2:60981)
0806 12:57:04.664740 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.664771 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 12:57:04.665417 32020 catalog_manager.cc:5495] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdbf455301784493b51959a559bb65d2 (127.24.83.2). New cstate: current_term: 1 leader_uuid: "bdbf455301784493b51959a559bb65d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdbf455301784493b51959a559bb65d2" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37379 } health_report { overall_health: HEALTHY } } }
I20260806 12:57:04.679210 32597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.677204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.24.83.2:37379)
0806 12:57:04.677263 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:04.677265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:04.677272 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:57:04.677300 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:04.677300 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:04.677316 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:04.677378 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:57:04.677469 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.677950 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.678633 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:04.678804 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:04.678852 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:04.678969 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:57:04.679059 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:57:04.679111 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:04.679147 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 12:57:04.679163 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 12:57:04.680266 32446 raft_consensus.cc:1261] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 1 FOLLOWER]: Refusing update from remote peer bdbf455301784493b51959a559bb65d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:04.680366 32160 raft_consensus.cc:1261] T 64723e16d378409695be5171e4a8b68e P a30d865941464d91bd03c5cc1cb2bee2 [term 1 FOLLOWER]: Refusing update from remote peer bdbf455301784493b51959a559bb65d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:04.680588 32606 consensus_queue.cc:1035] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45e4d5fb78f4ae4b94b9dbcbde1a08f" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:04.680714 32609 consensus_queue.cc:1035] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30d865941464d91bd03c5cc1cb2bee2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 37175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:04.681660 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.681449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.3:48245)
0806 12:57:04.681598 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.681632 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260806 12:57:04.681660 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.681567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.1:54423)
0806 12:57:04.681612 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.681632 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 12:57:04.682446 32613 mvcc.cc:205] Tried to move back new op lower bound from 7315542117088124928 to 7315542117026529280. Current Snapshot: MvccSnapshot[applied={T|T < 7315542117088124928}]
I20260806 12:57:04.682574 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.682275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.2:50891)
0806 12:57:04.682514 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.682545 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222}
I20260806 12:57:04.683224 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.683122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.1:58921)
0806 12:57:04.683175 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.683199 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 12:57:04.683244 32615 mvcc.cc:205] Tried to move back new op lower bound from 7315542117088124928 to 7315542117026529280. Current Snapshot: MvccSnapshot[applied={T|T < 7315542117088124928}]
I20260806 12:57:04.683346 32612 mvcc.cc:205] Tried to move back new op lower bound from 7315542117088124928 to 7315542117026529280. Current Snapshot: MvccSnapshot[applied={T|T < 7315542117088124928}]
I20260806 12:57:04.683763 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.683670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35127 (local address 127.24.83.3:47359)
0806 12:57:04.683704 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.683733 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 12:57:04.685267 32020 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:04.685360 32020 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:04.686547 32020 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63f73a41780e4913878e30fbdea6bc54: Sending DeleteTablet for 3 replicas of tablet 64723e16d378409695be5171e4a8b68e
I20260806 12:57:04.686808 32138 tablet_service.cc:1514] Processing DeleteTablet for tablet 64723e16d378409695be5171e4a8b68e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:04 UTC) from {username='oryx', principal='oryx/127.24.83.60@KRBTEST.COM'} at 127.0.0.1:56986
I20260806 12:57:04.686862 32424 tablet_service.cc:1514] Processing DeleteTablet for tablet 64723e16d378409695be5171e4a8b68e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:04 UTC) from {username='oryx', principal='oryx/127.24.83.60@KRBTEST.COM'} at 127.0.0.1:54452
I20260806 12:57:04.686879 32282 tablet_service.cc:1514] Processing DeleteTablet for tablet 64723e16d378409695be5171e4a8b68e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:04 UTC) from {username='oryx', principal='oryx/127.24.83.60@KRBTEST.COM'} at 127.0.0.1:48774
I20260806 12:57:04.687059 32624 tablet_replica.cc:330] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f: stopping tablet replica
I20260806 12:57:04.687119 32624 raft_consensus.cc:2229] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:04.687167 32624 raft_consensus.cc:2258] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:04.687273 32623 tablet_replica.cc:330] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2: stopping tablet replica
I20260806 12:57:04.687335 32623 raft_consensus.cc:2229] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:57:04.687414 32623 raft_consensus.cc:2258] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:04.687461 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 31974
I20260806 12:57:04.688524 32623 ts_tablet_manager.cc:1905] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:04.690310 32623 ts_tablet_manager.cc:1918] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:04.690375 32623 log.cc:1196] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/64723e16d378409695be5171e4a8b68e
I20260806 12:57:04.690629 32623 ts_tablet_manager.cc:1939] T 64723e16d378409695be5171e4a8b68e P bdbf455301784493b51959a559bb65d2: Deleting consensus metadata
I20260806 12:57:04.690935 32005 catalog_manager.cc:4841] TS bdbf455301784493b51959a559bb65d2 (127.24.83.2:37379): tablet 64723e16d378409695be5171e4a8b68e (table test-table [id=c2768637c1044ecbb77edb6db89c9912]) successfully deleted
I20260806 12:57:04.693416 32624 ts_tablet_manager.cc:1905] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:04.695047 32624 ts_tablet_manager.cc:1918] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:04.695116 32624 log.cc:1196] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/64723e16d378409695be5171e4a8b68e
I20260806 12:57:04.695385 32624 ts_tablet_manager.cc:1939] T 64723e16d378409695be5171e4a8b68e P a45e4d5fb78f4ae4b94b9dbcbde1a08f: Deleting consensus metadata
I20260806 12:57:04.695700 32007 catalog_manager.cc:4841] TS a45e4d5fb78f4ae4b94b9dbcbde1a08f (127.24.83.3:40855): tablet 64723e16d378409695be5171e4a8b68e (table test-table [id=c2768637c1044ecbb77edb6db89c9912]) successfully deleted
I20260806 12:57:04.699867 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:04.699745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:37175 (local address 127.0.0.1:57002)
0806 12:57:04.699802 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:04.699833 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 12:57:04.699999 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 32216
W20260806 12:57:04.701638 32005 catalog_manager.cc:4584] TS a30d865941464d91bd03c5cc1cb2bee2 (127.24.83.1:37175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64723e16d378409695be5171e4a8b68e: Network error: Client connection negotiation failed: client connection to 127.24.83.1:37175: connect: Connection refused (error 111)
I20260806 12:57:04.706005 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 32358
I20260806 12:57:04.711019 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 31683
I20260806 12:57:04.716029 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 31875
2026-08-06T12:57:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6593 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:57:04 dist-test-slave-zkpd krb5kdc[32627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:57:04 dist-test-slave-zkpd krb5kdc[32627](info): set up 2 sockets
Aug 06 12:57:04 dist-test-slave-zkpd krb5kdc[32627](info): commencing operation
krb5kdc: starting...
W20260806 12:57:05.758448 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:57:05 dist-test-slave-zkpd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:57:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:57:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.62.keytab.
Entry for principal oryx/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.62.keytab.
I20260806 12:57:05.871989 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40143
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.62.keytab
--principal=oryx/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:40143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:05.951617 32640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:05.951815 32640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:05.951836 32640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:05.952992 32640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:57:05.953037 32640 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 12:57:05.953049 32640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:05.953061 32640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:05.953073 32640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:57:05.953085 32640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:57:05.954612 32640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:40143
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.62.keytab
--principal=oryx/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:05.954866 32640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:05.955099 32640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:05.959839 32646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:05.959856 32648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:05.959839 32645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:05.960490 32640 server_base.cc:1034] running on GCE node
I20260806 12:57:05.960723 32640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:05.961059 32640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:05.962234 32640 hybrid_clock.cc:648] HybridClock initialized: now 1786021025962222 us; error 37 us; skew 500 ppm
Aug 06 12:57:05 dist-test-slave-zkpd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021025, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:05.968801 32640 init.cc:377] Logged in from keytab as oryx/127.24.83.62@KRBTEST.COM (short username oryx)
I20260806 12:57:05.969254 32640 webserver.cc:460] Webserver started at http://127.24.83.62:42987/ using document root <none> and password file <none>
I20260806 12:57:05.969481 32640 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:05.969530 32640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:05.969640 32640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:05.970768 32640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "75a391cae91a46f59ff1d95c3bb40e8f"
format_stamp: "Formatted at 2026-08-06 12:57:05 on dist-test-slave-zkpd"
server_key: "5140cb7b2d792e0366e0569f592af79b"
server_key_iv: "b89ef8c2aa602312b5f6c8503ed5704c"
server_key_version: "encryptionkey@0"
I20260806 12:57:05.971092 32640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "75a391cae91a46f59ff1d95c3bb40e8f"
format_stamp: "Formatted at 2026-08-06 12:57:05 on dist-test-slave-zkpd"
server_key: "5140cb7b2d792e0366e0569f592af79b"
server_key_iv: "b89ef8c2aa602312b5f6c8503ed5704c"
server_key_version: "encryptionkey@0"
I20260806 12:57:05.972559 32640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:57:05.973316 32655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:05.973495 32640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:05.973580 32640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "75a391cae91a46f59ff1d95c3bb40e8f"
format_stamp: "Formatted at 2026-08-06 12:57:05 on dist-test-slave-zkpd"
server_key: "5140cb7b2d792e0366e0569f592af79b"
server_key_iv: "b89ef8c2aa602312b5f6c8503ed5704c"
server_key_version: "encryptionkey@0"
I20260806 12:57:05.973644 32640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:05.998652 32640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:06.000502 32640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:06.000617 32640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:06.005033 32640 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:40143
I20260806 12:57:06.005187 32719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:40143 every 8 connection(s)
I20260806 12:57:06.005409 32640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:57:06.006116 32720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:06.006403 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 32640
I20260806 12:57:06.006481 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:57:06.006664 24908 external_mini_cluster.cc:1428] Setting key 7b6ae151075304294cca7cb57300ddb1
I20260806 12:57:06.009166 32720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f: Bootstrap starting.
I20260806 12:57:06.009912 32720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:06.010196 32720 log.cc:824] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:06.010869 32720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f: No bootstrap required, opened a new log
I20260806 12:57:06.012920 32720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a391cae91a46f59ff1d95c3bb40e8f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40143 } }
I20260806 12:57:06.013082 32720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:06.013145 32720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a391cae91a46f59ff1d95c3bb40e8f, State: Initialized, Role: FOLLOWER
I20260806 12:57:06.013294 32720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a391cae91a46f59ff1d95c3bb40e8f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40143 } }
I20260806 12:57:06.013377 32720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:57:06.013422 32720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:57:06.013468 32720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [term 0 FOLLOWER]: Advancing to term 1
Aug 06 12:57:06 dist-test-slave-zkpd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:57:06.014052 32720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a391cae91a46f59ff1d95c3bb40e8f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40143 } }
I20260806 12:57:06.014176 32720 leader_election.cc:302] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75a391cae91a46f59ff1d95c3bb40e8f; no voters:
I20260806 12:57:06.014380 32720 leader_election.cc:288] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:57:06.014451 32726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:06.014605 32726 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [term 1 LEADER]: Becoming Leader. State: Replica: 75a391cae91a46f59ff1d95c3bb40e8f, State: Running, Role: LEADER
I20260806 12:57:06.014674 32720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:57:06.014693 32726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a391cae91a46f59ff1d95c3bb40e8f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40143 } }
I20260806 12:57:06.015005 32726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75a391cae91a46f59ff1d95c3bb40e8f. Latest consensus state: current_term: 1 leader_uuid: "75a391cae91a46f59ff1d95c3bb40e8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a391cae91a46f59ff1d95c3bb40e8f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40143 } } }
I20260806 12:57:06.015097 32726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:06.015018 32727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75a391cae91a46f59ff1d95c3bb40e8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a391cae91a46f59ff1d95c3bb40e8f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40143 } } }
I20260806 12:57:06.015233 32727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:06.015311 32731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:57:06.015810 32731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:57:06.018132 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.007232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52978 (local address 127.24.83.62:40143)
0806 12:57:06.007441 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.007452 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.007489 (+ 37us) server_negotiation.cc:407] Connection header received
0806 12:57:06.008075 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.008084 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.008146 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.008299 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:06.008895 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.010282 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.011045 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.011469 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.011605 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.014143 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:06.014147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:06.014147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:06.014159 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:06.016495 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.016892 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.016895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.016895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.017019 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.017241 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.017242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.017242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.017483 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:06.017653 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.017734 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.017900 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260806 12:57:06.018422 32731 catalog_manager.cc:1347] Generated new cluster ID: bef444f294bf459fb728b64ef05aecef
I20260806 12:57:06.018471 32731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:57:06.033838 32731 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:57:06.034345 32731 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:57:06.039906 32731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f: Generated new TSK 0
I20260806 12:57:06.040081 32731 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab.
Entry for principal oryx/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab.
I20260806 12:57:06.072085 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40143
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab
--principal=oryx/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:06.189579 32747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:06.189846 32747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:06.189879 32747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:06.191534 32747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:06.191583 32747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:06.191612 32747 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:57:06.191648 32747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:57:06.193876 32747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab
--principal=oryx/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.83.62:40143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:06.194119 32747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:06.194370 32747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:06.199678 32753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:06.199831 32752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:06.199831 32755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:06.199985 32747 server_base.cc:1034] running on GCE node
I20260806 12:57:06.200220 32747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:06.200421 32747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:06.201553 32747 hybrid_clock.cc:648] HybridClock initialized: now 1786021026201538 us; error 30 us; skew 500 ppm
Aug 06 12:57:06 dist-test-slave-zkpd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021026, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:06.203999 32747 init.cc:377] Logged in from keytab as oryx/127.24.83.1@KRBTEST.COM (short username oryx)
I20260806 12:57:06.204387 32747 webserver.cc:460] Webserver started at http://127.24.83.1:35769/ using document root <none> and password file <none>
I20260806 12:57:06.204562 32747 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:06.204603 32747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:06.204700 32747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:06.205649 32747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "ca8117e7e3bc42949870a15d71e5b74c"
format_stamp: "Formatted at 2026-08-06 12:57:06 on dist-test-slave-zkpd"
server_key: "da7358abf5b74cd78c69c7361fa23465"
server_key_iv: "650b7d48bc49aedff9dd655bf6c40414"
server_key_version: "encryptionkey@0"
I20260806 12:57:06.205955 32747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "ca8117e7e3bc42949870a15d71e5b74c"
format_stamp: "Formatted at 2026-08-06 12:57:06 on dist-test-slave-zkpd"
server_key: "da7358abf5b74cd78c69c7361fa23465"
server_key_iv: "650b7d48bc49aedff9dd655bf6c40414"
server_key_version: "encryptionkey@0"
I20260806 12:57:06.207198 32747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 12:57:06.207940 32762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:06.208150 32747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:06.208246 32747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "ca8117e7e3bc42949870a15d71e5b74c"
format_stamp: "Formatted at 2026-08-06 12:57:06 on dist-test-slave-zkpd"
server_key: "da7358abf5b74cd78c69c7361fa23465"
server_key_iv: "650b7d48bc49aedff9dd655bf6c40414"
server_key_version: "encryptionkey@0"
I20260806 12:57:06.208323 32747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:06.223526 32747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:06.225337 32747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:06.225499 32747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:06.225983 32747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:06.226043 32747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:06.226075 32747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:06.226105 32747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:06.232985 32747 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:39747
Aug 06 12:57:06 dist-test-slave-zkpd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021026, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:57:06.233101 412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:39747 every 8 connection(s)
I20260806 12:57:06.233361 32747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:57:06.236387 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.226456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:53667 (local address 127.24.83.62:40143)
0806 12:57:06.226532 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.226536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.226690 (+ 154us) server_negotiation.cc:407] Connection header received
0806 12:57:06.227298 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.227300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.227313 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.227399 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:06.228229 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.229088 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.230552 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.230915 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.230997 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.233576 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:06.233578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:06.233579 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:06.233588 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:06.234996 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.235411 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.235412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.235413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.235472 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.235721 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.235722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.235722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.235839 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:06.235929 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.236199 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.236284 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:57:06.236536 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.226414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40143 (local address 127.24.83.1:53667)
0806 12:57:06.226562 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:06.226600 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:06.227160 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:06.227470 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:06.227474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:06.227803 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:06.228123 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.228129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.229182 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.229183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.230471 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.230474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.231009 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.231010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.231145 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.231665 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:06.231676 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:06.233478 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:06.235097 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.235105 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.235107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.235328 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.235541 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.235542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.235542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.235649 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.235921 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:06.235928 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:06.236063 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 12:57:06.236163 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 12:57:06.236316 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260806 12:57:06.237032 413 heartbeater.cc:347] Connected to a master server at 127.24.83.62:40143
I20260806 12:57:06.237138 413 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:06.237309 413 heartbeater.cc:511] Master 127.24.83.62:40143 requested a full tablet report, sending...
I20260806 12:57:06.237366 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 32747
I20260806 12:57:06.237493 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:57:06.237739 24908 external_mini_cluster.cc:1428] Setting key f0597281df9d66fda643ed1c35881e4f
I20260806 12:57:06.237856 32672 ts_manager.cc:194] Registered new tserver with Master: ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747)
I20260806 12:57:06.238984 32672 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:53667
I20260806 12:57:06.244628 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.238453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:41933 (local address 127.24.83.62:40143)
0806 12:57:06.238523 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.238525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.238532 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:57:06.238552 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.238553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.238563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.238625 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:06.238972 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.239679 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.240572 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.240866 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.240953 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.241943 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:06.241944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:06.241945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:06.241950 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:06.243438 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.243866 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.243867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.243868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.243956 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.244176 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.244177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.244177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.244271 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:06.244329 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.244510 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.244567 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 12:57:06.244752 420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.238065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40143 (local address 127.24.83.1:41933)
0806 12:57:06.238222 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:06.238232 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:06.238275 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:06.238640 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:06.238641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:06.238809 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:06.238874 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.238876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.239810 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.239811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.240491 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.240493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.240951 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.240952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.240998 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.241415 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:06.241421 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:06.241851 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:06.243523 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.243524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.243525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.243778 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.244033 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.244034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.244035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.244109 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.244338 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:06.244339 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:06.244406 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 12:57:06.244488 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 12:57:06.244604 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for oryx/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab.
Entry for principal oryx/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab.
I20260806 12:57:06.262737 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40143
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab
--principal=oryx/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:06.343663 423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:06.343864 423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:06.343904 423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:06.345036 423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:06.345077 423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:06.345098 423 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:57:06.345114 423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:57:06.346575 423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab
--principal=oryx/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.83.62:40143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:06.346758 423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:06.346922 423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:06.351681 432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:06.351660 429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:06.351783 423 server_base.cc:1034] running on GCE node
W20260806 12:57:06.351790 430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:06.352094 423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:06.352332 423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:06.353489 423 hybrid_clock.cc:648] HybridClock initialized: now 1786021026353482 us; error 32 us; skew 500 ppm
Aug 06 12:57:06 dist-test-slave-zkpd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021026, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:06.355882 423 init.cc:377] Logged in from keytab as oryx/127.24.83.2@KRBTEST.COM (short username oryx)
I20260806 12:57:06.356345 423 webserver.cc:460] Webserver started at http://127.24.83.2:33361/ using document root <none> and password file <none>
I20260806 12:57:06.356577 423 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:06.356657 423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:06.356787 423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:06.357752 423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "12b936d9c5c9436a8b1c01f2e870f359"
format_stamp: "Formatted at 2026-08-06 12:57:06 on dist-test-slave-zkpd"
server_key: "8fffaed4b97ca030b2d60ab443df1fe7"
server_key_iv: "527a463810171a26944052f3a57409fe"
server_key_version: "encryptionkey@0"
I20260806 12:57:06.358111 423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "12b936d9c5c9436a8b1c01f2e870f359"
format_stamp: "Formatted at 2026-08-06 12:57:06 on dist-test-slave-zkpd"
server_key: "8fffaed4b97ca030b2d60ab443df1fe7"
server_key_iv: "527a463810171a26944052f3a57409fe"
server_key_version: "encryptionkey@0"
I20260806 12:57:06.359401 423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 12:57:06.360137 440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:06.360301 423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:06.360388 423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "12b936d9c5c9436a8b1c01f2e870f359"
format_stamp: "Formatted at 2026-08-06 12:57:06 on dist-test-slave-zkpd"
server_key: "8fffaed4b97ca030b2d60ab443df1fe7"
server_key_iv: "527a463810171a26944052f3a57409fe"
server_key_version: "encryptionkey@0"
I20260806 12:57:06.360461 423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:06.367795 423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:06.369858 423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:06.370007 423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:06.370460 423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:06.370514 423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:06.370554 423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:06.370579 423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:57:06 dist-test-slave-zkpd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021026, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:57:06.377962 423 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:42441
I20260806 12:57:06.378047 559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:42441 every 8 connection(s)
I20260806 12:57:06.378311 423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:57:06.380494 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.370934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:46051 (local address 127.24.83.62:40143)
0806 12:57:06.370999 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.371002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.371218 (+ 216us) server_negotiation.cc:407] Connection header received
0806 12:57:06.371945 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.371947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.371960 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.372035 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:06.372935 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.373669 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.374694 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.374916 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.374992 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.377855 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:06.377857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:06.377858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:06.377866 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:06.379396 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.379730 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.379731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.379731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.379779 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.380015 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.380015 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.380015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.380110 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:06.380159 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.380373 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.380432 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 12:57:06.380692 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.370911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40143 (local address 127.24.83.2:46051)
0806 12:57:06.371083 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:06.371133 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:06.371800 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:06.372117 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:06.372122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:06.372510 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:06.372812 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.372818 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.373796 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.373798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.374600 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.374605 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.375010 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.375011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.375151 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.375647 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:06.375658 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:06.377756 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:06.379465 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.379468 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.379470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.379669 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.379824 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.379825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.379825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.379958 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.380162 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:06.380166 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:06.380268 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 12:57:06.380346 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:57:06.380466 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260806 12:57:06.381193 560 heartbeater.cc:347] Connected to a master server at 127.24.83.62:40143
I20260806 12:57:06.381273 560 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:06.381454 560 heartbeater.cc:511] Master 127.24.83.62:40143 requested a full tablet report, sending...
I20260806 12:57:06.381832 32671 ts_manager.cc:194] Registered new tserver with Master: 12b936d9c5c9436a8b1c01f2e870f359 (127.24.83.2:42441)
I20260806 12:57:06.382294 32671 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:46051
I20260806 12:57:06.387171 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.382200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:57789 (local address 127.24.83.62:40143)
0806 12:57:06.382417 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.382419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.382427 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:06.382446 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.382447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.382457 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.382507 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:06.382831 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.383279 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.383998 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.384192 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.384261 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.385019 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:06.385021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:06.385021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:06.385028 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:06.386054 (+ 1026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.386382 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.386384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.386384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.386437 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.386652 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.386653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.386653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.386791 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:06.386859 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.387017 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.387086 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199}
I20260806 12:57:06.387188 567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.382140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40143 (local address 127.24.83.2:57789)
0806 12:57:06.382253 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:06.382263 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:06.382311 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:06.382515 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:06.382516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:06.382702 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:06.382755 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.382756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.383374 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.383375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.383932 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.383934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.384261 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.384262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.384329 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.384669 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:06.384672 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:06.384935 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:06.386139 (+ 1204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.386140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.386141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.386303 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.386524 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.386525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.386525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.386587 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.386866 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:06.386867 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:06.386910 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 12:57:06.386989 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 12:57:06.387085 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":63,"threads_started":1}
I20260806 12:57:06.387190 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 423
I20260806 12:57:06.387310 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:57:06.387534 24908 external_mini_cluster.cc:1428] Setting key a5d584fe93568a1a98fc209e69f535cd
WARNING: no policy specified for oryx/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab.
Entry for principal oryx/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab.
I20260806 12:57:06.410223 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40143
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab
--principal=oryx/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:06.491034 570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:06.491217 570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:06.491238 570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:06.492390 570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:06.492429 570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:06.492449 570 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:57:06.492504 570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:57:06.493981 570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab
--principal=oryx/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.83.62:40143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:06.494203 570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:06.494429 570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:06.499130 578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:06.499167 576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:06.499171 575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:06.499430 570 server_base.cc:1034] running on GCE node
I20260806 12:57:06.499598 570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:06.499805 570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:06.500954 570 hybrid_clock.cc:648] HybridClock initialized: now 1786021026500950 us; error 34 us; skew 500 ppm
Aug 06 12:57:06 dist-test-slave-zkpd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021026, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:06.503502 570 init.cc:377] Logged in from keytab as oryx/127.24.83.3@KRBTEST.COM (short username oryx)
I20260806 12:57:06.503999 570 webserver.cc:460] Webserver started at http://127.24.83.3:33599/ using document root <none> and password file <none>
I20260806 12:57:06.504230 570 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:06.504273 570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:06.504422 570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:06.505373 570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "e52135d1b03a402b926b4d41be6962fa"
format_stamp: "Formatted at 2026-08-06 12:57:06 on dist-test-slave-zkpd"
server_key: "94e13a5a914f95663f05b52983087dd2"
server_key_iv: "82a7dce6fb24eacc6624c3ac0c43afa2"
server_key_version: "encryptionkey@0"
I20260806 12:57:06.505721 570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "e52135d1b03a402b926b4d41be6962fa"
format_stamp: "Formatted at 2026-08-06 12:57:06 on dist-test-slave-zkpd"
server_key: "94e13a5a914f95663f05b52983087dd2"
server_key_iv: "82a7dce6fb24eacc6624c3ac0c43afa2"
server_key_version: "encryptionkey@0"
I20260806 12:57:06.507072 570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:57:06.507738 585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:06.507929 570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:57:06.508016 570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "e52135d1b03a402b926b4d41be6962fa"
format_stamp: "Formatted at 2026-08-06 12:57:06 on dist-test-slave-zkpd"
server_key: "94e13a5a914f95663f05b52983087dd2"
server_key_iv: "82a7dce6fb24eacc6624c3ac0c43afa2"
server_key_version: "encryptionkey@0"
I20260806 12:57:06.508096 570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:06.543941 570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:06.545791 570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:06.545959 570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:06.546454 570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:06.546509 570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:06.546551 570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:06.546581 570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:57:06 dist-test-slave-zkpd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021026, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:57:06.553659 570 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:39017
I20260806 12:57:06.553949 704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:39017 every 8 connection(s)
I20260806 12:57:06.554095 570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:57:06.554651 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 570
I20260806 12:57:06.554754 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:57:06.554947 24908 external_mini_cluster.cc:1428] Setting key becb1070bb65bf4c152f9f03a92257f8
I20260806 12:57:06.556551 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.546895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:41895 (local address 127.24.83.62:40143)
0806 12:57:06.546961 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.546964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.547156 (+ 192us) server_negotiation.cc:407] Connection header received
0806 12:57:06.547621 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.547622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.547636 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.547716 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:06.548540 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.549384 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.550504 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.550735 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.550824 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.553417 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:06.553419 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:06.553419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:06.553431 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:06.555022 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.555562 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.555564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.555564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.555624 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.555931 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.555932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.555933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.556059 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:06.556130 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.556384 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.556463 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 12:57:06.556743 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.546880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40143 (local address 127.24.83.3:41895)
0806 12:57:06.547023 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:06.547061 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:06.547532 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:06.547788 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:06.547792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:06.548170 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:06.548453 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.548458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.549485 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.549486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.550402 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.550405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.550837 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.550838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.550959 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.551453 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:06.551461 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:06.553314 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:06.555192 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.555196 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.555198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.555504 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.555696 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.555697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.555698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.555826 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.556121 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:06.556127 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:06.556258 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 12:57:06.556351 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 12:57:06.556504 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260806 12:57:06.557201 705 heartbeater.cc:347] Connected to a master server at 127.24.83.62:40143
I20260806 12:57:06.557282 705 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:06.557464 705 heartbeater.cc:511] Master 127.24.83.62:40143 requested a full tablet report, sending...
I20260806 12:57:06.558087 32672 ts_manager.cc:194] Registered new tserver with Master: e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017)
I20260806 12:57:06.558486 32672 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:41895
I20260806 12:57:06.563397 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.558176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:51519 (local address 127.24.83.62:40143)
0806 12:57:06.558215 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.558217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.558288 (+ 71us) server_negotiation.cc:407] Connection header received
0806 12:57:06.558370 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.558371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.558379 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.558419 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:06.558764 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.559231 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.559980 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.560188 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.560263 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.561095 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:06.561096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:06.561097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:06.561103 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:06.562207 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.562528 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.562529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.562529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.562579 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.562828 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.562829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.562830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.562987 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:06.563058 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.563215 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.563281 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 12:57:06.563418 712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.558121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40143 (local address 127.24.83.3:51519)
0806 12:57:06.558237 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:06.558247 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:06.558314 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:06.558432 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:06.558433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:06.558639 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:06.558693 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.558695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.559329 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.559329 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.559857 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.559858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.560266 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.560267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.560334 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.560736 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:06.560741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:06.560996 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:06.562285 (+ 1289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.562294 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.562294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.562452 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.562644 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:06.562644 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:06.562645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:06.562701 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:06.563063 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:06.563064 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:06.563107 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 12:57:06.563186 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 12:57:06.563283 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
I20260806 12:57:06.566558 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 12:57:06 dist-test-slave-zkpd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021026, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 12:57:06 dist-test-slave-zkpd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:06 dist-test-slave-zkpd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 12:57:06.586971 32724 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:06.581804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.24.83.62:40143)
0806 12:57:06.581864 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.581867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.581880 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:57:06.581949 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.581950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.581960 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.582025 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:06.582644 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.583107 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.583844 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.584081 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.584150 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.586887 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52980: BlockingRecv error: recv got EOF from 127.0.0.1:52980 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 12:57:06 dist-test-slave-zkpd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021026, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:57:06.593918 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.587660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.24.83.62:40143)
0806 12:57:06.587724 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.587726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.587806 (+ 80us) server_negotiation.cc:407] Connection header received
0806 12:57:06.587881 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.587882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.587917 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.587962 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:06.588420 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.588864 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.589542 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.589733 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.589803 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.591664 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:06.591665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:06.591666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:06.591673 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:06.592813 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.593162 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.593163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.593163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.593216 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:06.593417 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:06.593418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:06.593418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:06.593522 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:06.593625 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.593740 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.593812 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:57:06.595319 32672 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:57:06.597077 32672 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:57:06.603662 729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.599243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:42441 (local address 127.0.0.1:41162)
0806 12:57:06.599543 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:06.599586 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:06.599674 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:06.600210 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:06.600210 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:06.600212 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:06.600262 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.600263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.601492 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.601493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.602452 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.602455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.603335 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.603337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.603436 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.603451 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 12:57:06.603521 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 12:57:06.603568 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":69,"threads_started":1}
I20260806 12:57:06.603700 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.599332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41162 (local address 127.24.83.2:42441)
0806 12:57:06.599510 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.599515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.599963 (+ 448us) server_negotiation.cc:407] Connection header received
0806 12:57:06.600017 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.600020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.600037 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.600114 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:06.600346 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.601407 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.602536 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.603237 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.603324 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.603403 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.603547 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.603587 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
I20260806 12:57:06.603756 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.599269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:39017 (local address 127.0.0.1:38842)
0806 12:57:06.600197 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:06.600208 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:06.600262 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:06.600437 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:06.600438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:06.600440 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:06.600489 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.600490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.601268 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.601269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.602650 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.602653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.603551 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.603552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.603602 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.603605 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:57:06.603645 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 12:57:06.603682 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":84,"thread_start_us":84,"threads_started":1}
I20260806 12:57:06.604101 730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.599260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:39747 (local address 127.0.0.1:56880)
0806 12:57:06.599795 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:06.599804 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:06.599869 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:06.600064 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:06.600067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:06.600068 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:06.600132 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.600133 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.601438 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.601439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.602650 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.602653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.603828 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.603829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.603886 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.603944 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 12:57:06.603995 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 12:57:06.604035 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260806 12:57:06.604159 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.599507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56880 (local address 127.24.83.1:39747)
0806 12:57:06.599693 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.599698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.599848 (+ 150us) server_negotiation.cc:407] Connection header received
0806 12:57:06.599959 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.599962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.599978 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.600055 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:06.600220 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.601355 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.602757 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.603245 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.603344 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.603424 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.604016 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.604059 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260806 12:57:06.604601 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.599503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38842 (local address 127.24.83.3:39017)
0806 12:57:06.599783 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.599786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.600256 (+ 470us) server_negotiation.cc:407] Connection header received
0806 12:57:06.600339 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.600341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.600353 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.600424 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:06.600541 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.601186 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.602737 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.603462 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.603548 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.603617 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.603653 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.603692 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":91,"threads_started":1}
I20260806 12:57:06.604717 491 tablet_service.cc:1467] Processing CreateTablet for tablet 52ae794c873b4c6bbe2e1906c37b92f3 (DEFAULT_TABLE table=test-table [id=ae86e85ddc6444a7b3919fb69bc7d10b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:06.604977 345 tablet_service.cc:1467] Processing CreateTablet for tablet 52ae794c873b4c6bbe2e1906c37b92f3 (DEFAULT_TABLE table=test-table [id=ae86e85ddc6444a7b3919fb69bc7d10b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:06.605106 491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52ae794c873b4c6bbe2e1906c37b92f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:06.605195 639 tablet_service.cc:1467] Processing CreateTablet for tablet 52ae794c873b4c6bbe2e1906c37b92f3 (DEFAULT_TABLE table=test-table [id=ae86e85ddc6444a7b3919fb69bc7d10b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:06.605319 345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52ae794c873b4c6bbe2e1906c37b92f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:06.605500 639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52ae794c873b4c6bbe2e1906c37b92f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:06.608844 734 tablet_bootstrap.cc:492] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: Bootstrap starting.
I20260806 12:57:06.609078 735 tablet_bootstrap.cc:492] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: Bootstrap starting.
I20260806 12:57:06.609508 734 tablet_bootstrap.cc:654] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:06.609737 735 tablet_bootstrap.cc:654] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:06.609858 734 log.cc:824] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:06.610056 735 log.cc:824] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:06.610188 736 tablet_bootstrap.cc:492] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: Bootstrap starting.
I20260806 12:57:06.610802 735 tablet_bootstrap.cc:492] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: No bootstrap required, opened a new log
I20260806 12:57:06.610807 736 tablet_bootstrap.cc:654] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:06.610883 735 ts_tablet_manager.cc:1397] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:57:06.610950 734 tablet_bootstrap.cc:492] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: No bootstrap required, opened a new log
I20260806 12:57:06.611027 734 ts_tablet_manager.cc:1397] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:57:06.611136 736 log.cc:824] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:06.612203 736 tablet_bootstrap.cc:492] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: No bootstrap required, opened a new log
I20260806 12:57:06.612277 736 ts_tablet_manager.cc:1397] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:57:06.612728 735 raft_consensus.cc:348] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:06.612877 735 raft_consensus.cc:374] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:06.612920 735 raft_consensus.cc:729] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12b936d9c5c9436a8b1c01f2e870f359, State: Initialized, Role: FOLLOWER
I20260806 12:57:06.613032 735 consensus_queue.cc:260] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:06.613004 734 raft_consensus.cc:348] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:06.613155 734 raft_consensus.cc:374] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:06.613193 734 raft_consensus.cc:729] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8117e7e3bc42949870a15d71e5b74c, State: Initialized, Role: FOLLOWER
I20260806 12:57:06.613310 735 ts_tablet_manager.cc:1428] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 12:57:06.613298 734 consensus_queue.cc:260] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:06.613436 560 heartbeater.cc:503] Master 127.24.83.62:40143 was elected leader, sending a full tablet report...
I20260806 12:57:06.613543 734 ts_tablet_manager.cc:1428] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 12:57:06.613605 413 heartbeater.cc:503] Master 127.24.83.62:40143 was elected leader, sending a full tablet report...
I20260806 12:57:06.614825 736 raft_consensus.cc:348] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:06.614948 736 raft_consensus.cc:374] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:06.614970 736 raft_consensus.cc:729] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e52135d1b03a402b926b4d41be6962fa, State: Initialized, Role: FOLLOWER
I20260806 12:57:06.615090 736 consensus_queue.cc:260] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:06.615343 736 ts_tablet_manager.cc:1428] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 12:57:06.615365 705 heartbeater.cc:503] Master 127.24.83.62:40143 was elected leader, sending a full tablet report...
W20260806 12:57:06.628801 562 tablet.cc:2369] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:57:06.734368 414 tablet.cc:2369] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:57:06.804713 706 tablet.cc:2369] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:57:06.991225 740 raft_consensus.cc:482] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:57:06.991358 740 raft_consensus.cc:504] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:06.991746 740 leader_election.cc:288] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747), e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017)
I20260806 12:57:06.995738 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.992103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:42867 (local address 127.24.83.3:39017)
0806 12:57:06.992151 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.992153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.992216 (+ 63us) server_negotiation.cc:407] Connection header received
0806 12:57:06.992301 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.992302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.992314 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.992382 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:06.992584 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.993329 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.994825 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.995375 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.995454 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.995464 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.995625 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.995664 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:57:06.995738 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.992016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:45887 (local address 127.24.83.1:39747)
0806 12:57:06.992084 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:06.992086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:06.992199 (+ 113us) server_negotiation.cc:407] Connection header received
0806 12:57:06.992312 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:06.992313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:06.992325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:06.992382 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:06.992583 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.993335 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.994824 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.995381 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.995454 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.995464 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:06.995625 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 12:57:06.995663 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:57:06.995775 744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.992029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:39017 (local address 127.24.83.2:42867)
0806 12:57:06.992138 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:06.992149 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:06.992215 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:06.992400 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:06.992401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:06.992404 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:06.992505 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.992507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.993416 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.993416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.994730 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.994733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.995467 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.995468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.995544 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.995550 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 12:57:06.995608 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 12:57:06.995658 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":97,"thread_start_us":58,"threads_started":1}
I20260806 12:57:06.995775 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:06.991962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:39747 (local address 127.24.83.2:45887)
0806 12:57:06.992135 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:06.992147 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:06.992215 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:06.992401 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:06.992401 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:06.992406 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:06.992505 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.992508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.993413 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.993414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.994729 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:06.994733 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:06.995466 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:06.995467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:06.995544 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:06.995549 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:57:06.995608 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 12:57:06.995658 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":50,"thread_start_us":86,"threads_started":1}
I20260806 12:57:06.996204 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ae794c873b4c6bbe2e1906c37b92f3" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52135d1b03a402b926b4d41be6962fa" is_pre_election: true
I20260806 12:57:06.996371 659 raft_consensus.cc:2454] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 0.
I20260806 12:57:06.996320 366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ae794c873b4c6bbe2e1906c37b92f3" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8117e7e3bc42949870a15d71e5b74c" is_pre_election: true
I20260806 12:57:06.996488 366 raft_consensus.cc:2454] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 0.
I20260806 12:57:06.996639 444 leader_election.cc:302] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b936d9c5c9436a8b1c01f2e870f359, e52135d1b03a402b926b4d41be6962fa; no voters:
I20260806 12:57:06.996830 740 raft_consensus.cc:2781] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:57:06.996914 740 raft_consensus.cc:482] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:57:06.996950 740 raft_consensus.cc:3037] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:06.997665 740 raft_consensus.cc:504] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:06.997826 740 leader_election.cc:288] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 election: Requested vote from peers ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747), e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017)
I20260806 12:57:06.997991 366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ae794c873b4c6bbe2e1906c37b92f3" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8117e7e3bc42949870a15d71e5b74c"
I20260806 12:57:06.998000 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ae794c873b4c6bbe2e1906c37b92f3" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52135d1b03a402b926b4d41be6962fa"
I20260806 12:57:06.998072 659 raft_consensus.cc:3037] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:06.998075 366 raft_consensus.cc:3037] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:06.998796 366 raft_consensus.cc:2454] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 1.
I20260806 12:57:06.998838 659 raft_consensus.cc:2454] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 1.
I20260806 12:57:06.999011 442 leader_election.cc:302] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b936d9c5c9436a8b1c01f2e870f359, ca8117e7e3bc42949870a15d71e5b74c; no voters:
I20260806 12:57:06.999101 740 raft_consensus.cc:2781] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:06.999195 740 raft_consensus.cc:686] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 LEADER]: Becoming Leader. State: Replica: 12b936d9c5c9436a8b1c01f2e870f359, State: Running, Role: LEADER
I20260806 12:57:06.999284 740 consensus_queue.cc:237] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.000135 32672 catalog_manager.cc:5495] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12b936d9c5c9436a8b1c01f2e870f359 (127.24.83.2). New cstate: current_term: 1 leader_uuid: "12b936d9c5c9436a8b1c01f2e870f359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } health_report { overall_health: UNKNOWN } } }
I20260806 12:57:07.041997 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.039609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.24.83.2:42441)
0806 12:57:07.039666 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:07.039668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:07.039678 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:57:07.039716 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:07.039717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:07.039738 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:07.039815 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:57:07.039962 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.040447 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.041213 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.041431 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.041506 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.041676 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:57:07.041778 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:57:07.041864 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:07.041890 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 12:57:07.041910 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 12:57:07.043934 659 raft_consensus.cc:1261] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Refusing update from remote peer 12b936d9c5c9436a8b1c01f2e870f359: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:07.043972 366 raft_consensus.cc:1261] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Refusing update from remote peer 12b936d9c5c9436a8b1c01f2e870f359: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:07.044308 745 consensus_queue.cc:1035] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:07.044456 740 consensus_queue.cc:1035] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:07.047492 749 mvcc.cc:205] Tried to move back new op lower bound from 7315542126768660480 to 7315542126589652992. Current Snapshot: MvccSnapshot[applied={T|T < 7315542126768660480}]
I20260806 12:57:07.047598 750 mvcc.cc:205] Tried to move back new op lower bound from 7315542126768660480 to 7315542126589652992. Current Snapshot: MvccSnapshot[applied={T|T < 7315542126768660480}]
I20260806 12:57:07.048714 32670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:07.048820 32670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:07.049526 32670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f: Sending DeleteTablet for 3 replicas of tablet 52ae794c873b4c6bbe2e1906c37b92f3
I20260806 12:57:07.049868 639 tablet_service.cc:1514] Processing DeleteTablet for tablet 52ae794c873b4c6bbe2e1906c37b92f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:38842
I20260806 12:57:07.049887 491 tablet_service.cc:1514] Processing DeleteTablet for tablet 52ae794c873b4c6bbe2e1906c37b92f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:41162
I20260806 12:57:07.050113 345 tablet_service.cc:1514] Processing DeleteTablet for tablet 52ae794c873b4c6bbe2e1906c37b92f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56880
I20260806 12:57:07.050307 760 tablet_replica.cc:330] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: stopping tablet replica
I20260806 12:57:07.050338 32670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 12:57:07.050410 760 raft_consensus.cc:2229] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:07.050416 758 tablet_replica.cc:330] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: stopping tablet replica
W20260806 12:57:07.050467 32670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:57:07.050506 760 raft_consensus.cc:2258] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.050506 758 raft_consensus.cc:2229] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:07.050612 758 raft_consensus.cc:2258] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.051049 760 ts_tablet_manager.cc:1905] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:07.051688 345 tablet_service.cc:1467] Processing CreateTablet for tablet a9f26e336f6942488b40bf72ba11b417 (DEFAULT_TABLE table=test-table [id=b1b8c63c37304aee85d4d0959b40b6f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:07.051752 639 tablet_service.cc:1467] Processing CreateTablet for tablet a9f26e336f6942488b40bf72ba11b417 (DEFAULT_TABLE table=test-table [id=b1b8c63c37304aee85d4d0959b40b6f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:07.051806 345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9f26e336f6942488b40bf72ba11b417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.051777 491 tablet_service.cc:1467] Processing CreateTablet for tablet a9f26e336f6942488b40bf72ba11b417 (DEFAULT_TABLE table=test-table [id=b1b8c63c37304aee85d4d0959b40b6f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:07.051843 639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9f26e336f6942488b40bf72ba11b417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.051926 491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9f26e336f6942488b40bf72ba11b417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.053134 734 tablet_bootstrap.cc:492] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c: Bootstrap starting.
I20260806 12:57:07.053155 736 tablet_bootstrap.cc:492] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa: Bootstrap starting.
I20260806 12:57:07.053215 735 tablet_bootstrap.cc:492] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359: Bootstrap starting.
I20260806 12:57:07.053553 734 tablet_bootstrap.cc:654] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.053567 736 tablet_bootstrap.cc:654] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.053653 735 tablet_bootstrap.cc:654] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.053755 759 tablet_replica.cc:330] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: stopping tablet replica
I20260806 12:57:07.053853 759 raft_consensus.cc:2229] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:57:07.053947 759 raft_consensus.cc:2258] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.054386 759 ts_tablet_manager.cc:1905] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:07.054445 758 ts_tablet_manager.cc:1905] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:07.055289 760 ts_tablet_manager.cc:1918] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:07.055369 760 log.cc:1196] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/wal/wals/52ae794c873b4c6bbe2e1906c37b92f3
I20260806 12:57:07.055614 760 ts_tablet_manager.cc:1939] T 52ae794c873b4c6bbe2e1906c37b92f3 P ca8117e7e3bc42949870a15d71e5b74c: Deleting consensus metadata
I20260806 12:57:07.055876 758 ts_tablet_manager.cc:1918] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:07.055968 758 log.cc:1196] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/52ae794c873b4c6bbe2e1906c37b92f3
I20260806 12:57:07.056072 759 ts_tablet_manager.cc:1918] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:07.056072 32657 catalog_manager.cc:4841] TS ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747): tablet 52ae794c873b4c6bbe2e1906c37b92f3 (table test-table [id=ae86e85ddc6444a7b3919fb69bc7d10b]) successfully deleted
I20260806 12:57:07.056208 759 log.cc:1196] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/52ae794c873b4c6bbe2e1906c37b92f3
I20260806 12:57:07.056224 758 ts_tablet_manager.cc:1939] T 52ae794c873b4c6bbe2e1906c37b92f3 P e52135d1b03a402b926b4d41be6962fa: Deleting consensus metadata
I20260806 12:57:07.056480 759 ts_tablet_manager.cc:1939] T 52ae794c873b4c6bbe2e1906c37b92f3 P 12b936d9c5c9436a8b1c01f2e870f359: Deleting consensus metadata
I20260806 12:57:07.056715 736 tablet_bootstrap.cc:492] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa: No bootstrap required, opened a new log
I20260806 12:57:07.056761 736 ts_tablet_manager.cc:1397] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 12:57:07.056957 32659 catalog_manager.cc:4841] TS e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017): tablet 52ae794c873b4c6bbe2e1906c37b92f3 (table test-table [id=ae86e85ddc6444a7b3919fb69bc7d10b]) successfully deleted
I20260806 12:57:07.056938 736 raft_consensus.cc:348] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.057014 736 raft_consensus.cc:374] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.057040 736 raft_consensus.cc:729] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e52135d1b03a402b926b4d41be6962fa, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.057062 32656 catalog_manager.cc:4841] TS 12b936d9c5c9436a8b1c01f2e870f359 (127.24.83.2:42441): tablet 52ae794c873b4c6bbe2e1906c37b92f3 (table test-table [id=ae86e85ddc6444a7b3919fb69bc7d10b]) successfully deleted
I20260806 12:57:07.057083 736 consensus_queue.cc:260] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.057173 736 ts_tablet_manager.cc:1428] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:07.057495 734 tablet_bootstrap.cc:492] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c: No bootstrap required, opened a new log
I20260806 12:57:07.057521 735 tablet_bootstrap.cc:492] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359: No bootstrap required, opened a new log
I20260806 12:57:07.057547 734 ts_tablet_manager.cc:1397] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 12:57:07.057562 735 ts_tablet_manager.cc:1397] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 12:57:07.057700 735 raft_consensus.cc:348] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.057775 735 raft_consensus.cc:374] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.057827 735 raft_consensus.cc:729] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12b936d9c5c9436a8b1c01f2e870f359, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.057886 735 consensus_queue.cc:260] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.057878 734 raft_consensus.cc:348] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.057962 734 raft_consensus.cc:374] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.057987 734 raft_consensus.cc:729] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8117e7e3bc42949870a15d71e5b74c, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.057992 735 ts_tablet_manager.cc:1428] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:07.058027 734 consensus_queue.cc:260] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.058131 734 ts_tablet_manager.cc:1428] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:07.089490 754 raft_consensus.cc:482] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:57:07.089576 754 raft_consensus.cc:504] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.089748 754 leader_election.cc:288] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747), e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017)
I20260806 12:57:07.089901 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9f26e336f6942488b40bf72ba11b417" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52135d1b03a402b926b4d41be6962fa" is_pre_election: true
I20260806 12:57:07.089915 366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9f26e336f6942488b40bf72ba11b417" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8117e7e3bc42949870a15d71e5b74c" is_pre_election: true
I20260806 12:57:07.089988 366 raft_consensus.cc:2454] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 0.
I20260806 12:57:07.090003 659 raft_consensus.cc:2454] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 0.
I20260806 12:57:07.090142 442 leader_election.cc:302] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b936d9c5c9436a8b1c01f2e870f359, ca8117e7e3bc42949870a15d71e5b74c; no voters:
I20260806 12:57:07.090238 754 raft_consensus.cc:2781] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:57:07.090286 754 raft_consensus.cc:482] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:57:07.090306 754 raft_consensus.cc:3037] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.091112 754 raft_consensus.cc:504] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.091238 754 leader_election.cc:288] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 election: Requested vote from peers ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747), e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017)
I20260806 12:57:07.091393 366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9f26e336f6942488b40bf72ba11b417" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8117e7e3bc42949870a15d71e5b74c"
I20260806 12:57:07.091452 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9f26e336f6942488b40bf72ba11b417" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52135d1b03a402b926b4d41be6962fa"
I20260806 12:57:07.091480 366 raft_consensus.cc:3037] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.091507 659 raft_consensus.cc:3037] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.092186 659 raft_consensus.cc:2454] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 1.
I20260806 12:57:07.092186 366 raft_consensus.cc:2454] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 1.
I20260806 12:57:07.092343 444 leader_election.cc:302] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b936d9c5c9436a8b1c01f2e870f359, e52135d1b03a402b926b4d41be6962fa; no voters:
I20260806 12:57:07.092449 754 raft_consensus.cc:2781] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:07.092514 754 raft_consensus.cc:686] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 LEADER]: Becoming Leader. State: Replica: 12b936d9c5c9436a8b1c01f2e870f359, State: Running, Role: LEADER
I20260806 12:57:07.092561 754 consensus_queue.cc:237] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.093083 32670 catalog_manager.cc:5495] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12b936d9c5c9436a8b1c01f2e870f359 (127.24.83.2). New cstate: current_term: 1 leader_uuid: "12b936d9c5c9436a8b1c01f2e870f359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } health_report { overall_health: UNKNOWN } } }
Aug 06 12:57:07 dist-test-slave-zkpd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021025, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.62@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:57:07.127524 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.120576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52996 (local address 127.24.83.62:40143)
0806 12:57:07.120683 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:07.120685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:07.120694 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:57:07.120768 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:07.120768 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:07.120780 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:07.120865 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:07.121320 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.121818 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.122490 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.122678 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.122754 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.124900 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:07.124901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:07.124901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:07.124908 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:07.126340 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:07.126720 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:07.126721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:07.126722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:07.126786 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:07.126970 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:07.126971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:07.126972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:07.127081 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:07.127155 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:07.127301 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 12:57:07.127388 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":71,"threads_started":1}
I20260806 12:57:07.127588 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.120517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40143 (local address 127.0.0.1:52996)
0806 12:57:07.120631 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:07.120643 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:07.120707 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:07.120868 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:07.120869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:07.121189 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:07.121231 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:07.121234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.121901 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.121902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:07.122416 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:07.122417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.122754 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.122755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:07.122827 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.123406 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:07.123410 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:07.124806 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:07.126436 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:07.126445 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:07.126447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:07.126638 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:07.126849 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:07.126850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:07.126850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:07.126910 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:07.127143 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:07.127151 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:07.127196 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 12:57:07.127279 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 12:57:07.127454 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":76,"thread_start_us":61,"threads_started":1}
I20260806 12:57:07.129808 32670 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:52996:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 12:57:07.130030 32670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:57:07.131556 491 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3c15cfd5f1457c963a3766a5987b90 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7228b321794b4918b4e9f5535eab713b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:57:07.131649 345 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3c15cfd5f1457c963a3766a5987b90 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7228b321794b4918b4e9f5535eab713b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:57:07.131640 639 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3c15cfd5f1457c963a3766a5987b90 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7228b321794b4918b4e9f5535eab713b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:57:07.131721 491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3c15cfd5f1457c963a3766a5987b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.131774 345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3c15cfd5f1457c963a3766a5987b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.131814 639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3c15cfd5f1457c963a3766a5987b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.133204 735 tablet_bootstrap.cc:492] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: Bootstrap starting.
I20260806 12:57:07.133204 734 tablet_bootstrap.cc:492] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c: Bootstrap starting.
I20260806 12:57:07.133641 734 tablet_bootstrap.cc:654] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.133641 735 tablet_bootstrap.cc:654] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.134020 736 tablet_bootstrap.cc:492] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa: Bootstrap starting.
I20260806 12:57:07.134402 735 tablet_bootstrap.cc:492] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: No bootstrap required, opened a new log
I20260806 12:57:07.134408 736 tablet_bootstrap.cc:654] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.134459 735 ts_tablet_manager.cc:1397] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:07.134635 735 raft_consensus.cc:348] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.134717 735 raft_consensus.cc:374] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.134758 735 raft_consensus.cc:729] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12b936d9c5c9436a8b1c01f2e870f359, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.134814 735 consensus_queue.cc:260] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.134924 735 ts_tablet_manager.cc:1428] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:07.134984 754 tablet_replica.cc:439] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } } }
I20260806 12:57:07.135056 754 tablet_replica.cc:442] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:57:07.135672 736 tablet_bootstrap.cc:492] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa: No bootstrap required, opened a new log
I20260806 12:57:07.135725 736 ts_tablet_manager.cc:1397] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:57:07.135881 736 raft_consensus.cc:348] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.135982 736 raft_consensus.cc:374] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.136003 736 raft_consensus.cc:729] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e52135d1b03a402b926b4d41be6962fa, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.136049 736 consensus_queue.cc:260] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.136142 736 ts_tablet_manager.cc:1428] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:07.136260 742 tablet_replica.cc:439] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } } }
I20260806 12:57:07.136330 742 tablet_replica.cc:442] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:57:07.136341 734 tablet_bootstrap.cc:492] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c: No bootstrap required, opened a new log
I20260806 12:57:07.136394 734 ts_tablet_manager.cc:1397] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 12:57:07.136582 734 raft_consensus.cc:348] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.136650 734 raft_consensus.cc:374] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.136684 734 raft_consensus.cc:729] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8117e7e3bc42949870a15d71e5b74c, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.136724 734 consensus_queue.cc:260] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.136811 734 ts_tablet_manager.cc:1428] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:07.136863 741 tablet_replica.cc:439] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } } }
I20260806 12:57:07.136962 741 tablet_replica.cc:442] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:57:07.177357 754 raft_consensus.cc:482] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:57:07.177455 754 raft_consensus.cc:504] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.177654 754 leader_election.cc:288] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747), e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017)
I20260806 12:57:07.177838 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3c15cfd5f1457c963a3766a5987b90" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52135d1b03a402b926b4d41be6962fa" is_pre_election: true
I20260806 12:57:07.177835 366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3c15cfd5f1457c963a3766a5987b90" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8117e7e3bc42949870a15d71e5b74c" is_pre_election: true
I20260806 12:57:07.177961 659 raft_consensus.cc:2454] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 0.
I20260806 12:57:07.177959 366 raft_consensus.cc:2454] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 0.
I20260806 12:57:07.178123 444 leader_election.cc:302] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b936d9c5c9436a8b1c01f2e870f359, e52135d1b03a402b926b4d41be6962fa; no voters:
I20260806 12:57:07.178244 754 raft_consensus.cc:2781] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:57:07.178292 754 raft_consensus.cc:482] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:57:07.178316 754 raft_consensus.cc:3037] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.178951 754 raft_consensus.cc:504] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.179087 754 leader_election.cc:288] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 election: Requested vote from peers ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747), e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017)
I20260806 12:57:07.179226 366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3c15cfd5f1457c963a3766a5987b90" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8117e7e3bc42949870a15d71e5b74c"
I20260806 12:57:07.179240 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3c15cfd5f1457c963a3766a5987b90" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52135d1b03a402b926b4d41be6962fa"
I20260806 12:57:07.179306 659 raft_consensus.cc:3037] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.179307 366 raft_consensus.cc:3037] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.179942 659 raft_consensus.cc:2454] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 1.
I20260806 12:57:07.180020 366 raft_consensus.cc:2454] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 1.
I20260806 12:57:07.180119 444 leader_election.cc:302] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b936d9c5c9436a8b1c01f2e870f359, e52135d1b03a402b926b4d41be6962fa; no voters:
I20260806 12:57:07.180209 754 raft_consensus.cc:2781] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:07.180272 754 raft_consensus.cc:686] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 LEADER]: Becoming Leader. State: Replica: 12b936d9c5c9436a8b1c01f2e870f359, State: Running, Role: LEADER
I20260806 12:57:07.180327 754 consensus_queue.cc:237] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.180613 740 tablet_replica.cc:439] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: TxnStatusTablet state changed. Reason: New leader 12b936d9c5c9436a8b1c01f2e870f359. Latest consensus state: current_term: 1 leader_uuid: "12b936d9c5c9436a8b1c01f2e870f359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } } }
I20260806 12:57:07.180718 740 tablet_replica.cc:442] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:57:07.180903 776 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 12:57:07.180908 32670 catalog_manager.cc:5495] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12b936d9c5c9436a8b1c01f2e870f359 (127.24.83.2). New cstate: current_term: 1 leader_uuid: "12b936d9c5c9436a8b1c01f2e870f359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } health_report { overall_health: UNKNOWN } } }
I20260806 12:57:07.181012 776 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 12:57:07.194228 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.191702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41170 (local address 127.24.83.2:42441)
0806 12:57:07.191754 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:07.191756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:07.191783 (+ 27us) server_negotiation.cc:407] Connection header received
0806 12:57:07.191919 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:07.191920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:07.191930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:07.191976 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:57:07.192106 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.192559 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.193640 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.193820 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.193868 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.194046 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:57:07.194069 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:57:07.194105 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:07.194161 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 12:57:07.194178 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 12:57:07.194288 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.191654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:42441 (local address 127.0.0.1:41170)
0806 12:57:07.191739 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:07.191750 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:07.191809 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:07.191994 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:07.191995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:07.191997 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:57:07.192049 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:07.192052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.192625 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.192626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:07.193564 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:07.193566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.193886 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.193887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:07.193953 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.193971 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:57:07.194106 (+ 135us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:57:07.194111 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:57:07.194160 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 12:57:07.194208 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 12:57:07.195140 659 raft_consensus.cc:1261] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Refusing update from remote peer 12b936d9c5c9436a8b1c01f2e870f359: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:07.195174 366 raft_consensus.cc:1261] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Refusing update from remote peer 12b936d9c5c9436a8b1c01f2e870f359: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:07.195371 740 consensus_queue.cc:1035] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:07.195468 754 consensus_queue.cc:1035] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:07.195876 742 tablet_replica.cc:439] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa: TxnStatusTablet state changed. Reason: New leader 12b936d9c5c9436a8b1c01f2e870f359. Latest consensus state: current_term: 1 leader_uuid: "12b936d9c5c9436a8b1c01f2e870f359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } } }
I20260806 12:57:07.195977 742 tablet_replica.cc:442] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:57:07.196592 741 tablet_replica.cc:439] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c: TxnStatusTablet state changed. Reason: New leader 12b936d9c5c9436a8b1c01f2e870f359. Latest consensus state: current_term: 1 leader_uuid: "12b936d9c5c9436a8b1c01f2e870f359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } } }
I20260806 12:57:07.196695 741 tablet_replica.cc:442] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:57:07.196931 754 tablet_replica.cc:439] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "12b936d9c5c9436a8b1c01f2e870f359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } } }
I20260806 12:57:07.197005 754 tablet_replica.cc:442] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:57:07.197211 748 mvcc.cc:205] Tried to move back new op lower bound from 7315542127389806592 to 7315542127330996224. Current Snapshot: MvccSnapshot[applied={T|T < 7315542127389806592 or (T in {7315542127389806592})}]
I20260806 12:57:07.197553 742 tablet_replica.cc:439] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "12b936d9c5c9436a8b1c01f2e870f359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } } }
I20260806 12:57:07.197638 742 tablet_replica.cc:442] T 6e3c15cfd5f1457c963a3766a5987b90 P e52135d1b03a402b926b4d41be6962fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:57:07.198017 745 tablet_replica.cc:439] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "12b936d9c5c9436a8b1c01f2e870f359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } } }
I20260806 12:57:07.198119 745 tablet_replica.cc:442] T 6e3c15cfd5f1457c963a3766a5987b90 P 12b936d9c5c9436a8b1c01f2e870f359: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:57:07.198993 741 tablet_replica.cc:439] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "12b936d9c5c9436a8b1c01f2e870f359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } } }
I20260806 12:57:07.199079 741 tablet_replica.cc:442] T 6e3c15cfd5f1457c963a3766a5987b90 P ca8117e7e3bc42949870a15d71e5b74c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:57:07.203148 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.200554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:48313 (local address 127.24.83.2:42441)
0806 12:57:07.200592 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:07.200593 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:07.200729 (+ 136us) server_negotiation.cc:407] Connection header received
0806 12:57:07.200802 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:07.200803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:07.200813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:07.200864 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:57:07.200973 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.201670 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.202444 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.202734 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.202805 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.202932 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:57:07.202963 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:57:07.203011 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:07.203054 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 12:57:07.203072 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 12:57:07.203148 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.200492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:42441 (local address 127.24.83.2:48313)
0806 12:57:07.200654 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:07.200662 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:07.200729 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:07.200873 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:07.200873 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:07.200875 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:57:07.200916 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:07.200917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.201746 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.201747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:07.202371 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:07.202373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.202809 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.202809 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:07.202847 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.202859 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:57:07.203017 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:57:07.203018 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 12:57:07.203052 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 12:57:07.203083 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260806 12:57:07.204916 659 raft_consensus.cc:1261] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Refusing update from remote peer 12b936d9c5c9436a8b1c01f2e870f359: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:07.204957 366 raft_consensus.cc:1261] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Refusing update from remote peer 12b936d9c5c9436a8b1c01f2e870f359: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:07.205132 745 consensus_queue.cc:1035] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:07.205210 754 consensus_queue.cc:1035] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:07.217999 32670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:07.218092 32670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:07.218650 32670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f: Sending DeleteTablet for 3 replicas of tablet a9f26e336f6942488b40bf72ba11b417
I20260806 12:57:07.218904 491 tablet_service.cc:1514] Processing DeleteTablet for tablet a9f26e336f6942488b40bf72ba11b417 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:41162
I20260806 12:57:07.218916 639 tablet_service.cc:1514] Processing DeleteTablet for tablet a9f26e336f6942488b40bf72ba11b417 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:38842
I20260806 12:57:07.218992 759 tablet_replica.cc:330] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359: stopping tablet replica
I20260806 12:57:07.219000 758 tablet_replica.cc:330] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa: stopping tablet replica
I20260806 12:57:07.219040 758 raft_consensus.cc:2229] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:07.219041 759 raft_consensus.cc:2229] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:57:07.219025 345 tablet_service.cc:1514] Processing DeleteTablet for tablet a9f26e336f6942488b40bf72ba11b417 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56880
I20260806 12:57:07.219072 758 raft_consensus.cc:2258] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.219095 759 raft_consensus.cc:2258] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.219162 760 tablet_replica.cc:330] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c: stopping tablet replica
I20260806 12:57:07.219204 760 raft_consensus.cc:2229] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:07.219236 760 raft_consensus.cc:2258] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.219358 758 ts_tablet_manager.cc:1905] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:07.219583 759 ts_tablet_manager.cc:1905] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:07.221189 758 ts_tablet_manager.cc:1918] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:57:07.221264 758 log.cc:1196] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/a9f26e336f6942488b40bf72ba11b417
I20260806 12:57:07.221484 758 ts_tablet_manager.cc:1939] T a9f26e336f6942488b40bf72ba11b417 P e52135d1b03a402b926b4d41be6962fa: Deleting consensus metadata
I20260806 12:57:07.221696 759 ts_tablet_manager.cc:1918] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:57:07.221751 759 log.cc:1196] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/a9f26e336f6942488b40bf72ba11b417
I20260806 12:57:07.221779 32659 catalog_manager.cc:4841] TS e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017): tablet a9f26e336f6942488b40bf72ba11b417 (table test-table [id=b1b8c63c37304aee85d4d0959b40b6f1]) successfully deleted
I20260806 12:57:07.221976 759 ts_tablet_manager.cc:1939] T a9f26e336f6942488b40bf72ba11b417 P 12b936d9c5c9436a8b1c01f2e870f359: Deleting consensus metadata
I20260806 12:57:07.222271 32656 catalog_manager.cc:4841] TS 12b936d9c5c9436a8b1c01f2e870f359 (127.24.83.2:42441): tablet a9f26e336f6942488b40bf72ba11b417 (table test-table [id=b1b8c63c37304aee85d4d0959b40b6f1]) successfully deleted
I20260806 12:57:07.222568 760 ts_tablet_manager.cc:1905] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:07.224112 760 ts_tablet_manager.cc:1918] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:57:07.224176 760 log.cc:1196] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/wal/wals/a9f26e336f6942488b40bf72ba11b417
I20260806 12:57:07.224386 760 ts_tablet_manager.cc:1939] T a9f26e336f6942488b40bf72ba11b417 P ca8117e7e3bc42949870a15d71e5b74c: Deleting consensus metadata
I20260806 12:57:07.224691 32657 catalog_manager.cc:4841] TS ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747): tablet a9f26e336f6942488b40bf72ba11b417 (table test-table [id=b1b8c63c37304aee85d4d0959b40b6f1]) successfully deleted
Aug 06 12:57:07 dist-test-slave-zkpd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 12:57:07 dist-test-slave-zkpd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:07 dist-test-slave-zkpd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 12:57:07.245133 771 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:07.238934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.24.83.62:40143)
0806 12:57:07.238979 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:07.238982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:07.239143 (+ 161us) server_negotiation.cc:407] Connection header received
0806 12:57:07.239208 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:07.239209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:07.239221 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:07.239275 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:07.239799 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.240676 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.241413 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.241729 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.241794 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.245069 (+ 3275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52998: BlockingRecv error: recv got EOF from 127.0.0.1:52998 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 12:57:07 dist-test-slave-zkpd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM
I20260806 12:57:07.253118 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.245904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53002 (local address 127.24.83.62:40143)
0806 12:57:07.245944 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:07.245946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:07.246036 (+ 90us) server_negotiation.cc:407] Connection header received
0806 12:57:07.246094 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:07.246095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:07.246104 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:07.246145 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:07.246718 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.247147 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.248164 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.248348 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.248402 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.250620 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:07.250622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:07.250622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:07.250630 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:07.252041 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:07.252381 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:07.252383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:07.252383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:07.252442 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:07.252628 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:07.252629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:07.252629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:07.252736 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:07.252838 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:07.252944 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 12:57:07.253022 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:57:07.254709 32670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:57:07.254874 32670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:57:07.256215 345 tablet_service.cc:1467] Processing CreateTablet for tablet 75150cc75c294fec8b8e205c00c6b720 (DEFAULT_TABLE table=test-table [id=eb5f0d1914e745c88e8480b5abded72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:07.256327 491 tablet_service.cc:1467] Processing CreateTablet for tablet 75150cc75c294fec8b8e205c00c6b720 (DEFAULT_TABLE table=test-table [id=eb5f0d1914e745c88e8480b5abded72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:07.256367 345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75150cc75c294fec8b8e205c00c6b720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.256426 491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75150cc75c294fec8b8e205c00c6b720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.256412 639 tablet_service.cc:1467] Processing CreateTablet for tablet 75150cc75c294fec8b8e205c00c6b720 (DEFAULT_TABLE table=test-table [id=eb5f0d1914e745c88e8480b5abded72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:07.256536 639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75150cc75c294fec8b8e205c00c6b720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.257776 736 tablet_bootstrap.cc:492] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa: Bootstrap starting.
I20260806 12:57:07.257808 734 tablet_bootstrap.cc:492] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c: Bootstrap starting.
I20260806 12:57:07.258174 736 tablet_bootstrap.cc:654] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.258206 734 tablet_bootstrap.cc:654] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.258555 735 tablet_bootstrap.cc:492] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359: Bootstrap starting.
I20260806 12:57:07.258853 735 tablet_bootstrap.cc:654] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.258952 736 tablet_bootstrap.cc:492] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa: No bootstrap required, opened a new log
I20260806 12:57:07.259017 736 ts_tablet_manager.cc:1397] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:07.259169 736 raft_consensus.cc:348] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.259254 736 raft_consensus.cc:374] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.259284 736 raft_consensus.cc:729] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e52135d1b03a402b926b4d41be6962fa, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.259331 736 consensus_queue.cc:260] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.259440 736 ts_tablet_manager.cc:1428] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:07.259486 734 tablet_bootstrap.cc:492] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c: No bootstrap required, opened a new log
I20260806 12:57:07.259541 734 ts_tablet_manager.cc:1397] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:57:07.259711 734 raft_consensus.cc:348] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.259790 734 raft_consensus.cc:374] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.259817 734 raft_consensus.cc:729] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8117e7e3bc42949870a15d71e5b74c, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.259862 734 consensus_queue.cc:260] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.259940 735 tablet_bootstrap.cc:492] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359: No bootstrap required, opened a new log
I20260806 12:57:07.259980 735 ts_tablet_manager.cc:1397] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:07.259995 734 ts_tablet_manager.cc:1428] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:07.260077 735 raft_consensus.cc:348] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.260146 735 raft_consensus.cc:374] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.260164 735 raft_consensus.cc:729] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12b936d9c5c9436a8b1c01f2e870f359, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.260206 735 consensus_queue.cc:260] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.260293 735 ts_tablet_manager.cc:1428] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:07.336704 741 raft_consensus.cc:482] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:57:07.336820 741 raft_consensus.cc:504] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.337193 741 leader_election.cc:288] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12b936d9c5c9436a8b1c01f2e870f359 (127.24.83.2:42441), e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017)
I20260806 12:57:07.340968 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.337457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:45457 (local address 127.24.83.3:39017)
0806 12:57:07.337543 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:07.337546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:07.337641 (+ 95us) server_negotiation.cc:407] Connection header received
0806 12:57:07.337701 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:07.337702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:07.337712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:07.337765 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:07.337964 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.338638 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.340082 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.340607 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.340676 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.340685 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:07.340862 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 12:57:07.340899 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:57:07.340968 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.337455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:55379 (local address 127.24.83.2:42441)
0806 12:57:07.337503 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:07.337505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:07.337643 (+ 138us) server_negotiation.cc:407] Connection header received
0806 12:57:07.337700 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:07.337701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:07.337712 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:07.337765 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:07.338294 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.338914 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.340089 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.340604 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.340676 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.340688 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:07.340863 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 12:57:07.340898 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 12:57:07.341012 803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.337392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:42441 (local address 127.24.83.1:55379)
0806 12:57:07.337566 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:07.337577 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:07.337636 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:07.337775 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:07.337776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:07.337780 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:07.338203 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:07.338206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.338984 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.338985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:07.340021 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:07.340023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.340710 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.340711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:07.340783 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.340787 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:57:07.340843 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 12:57:07.340891 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":46,"thread_start_us":91,"threads_started":1}
I20260806 12:57:07.341012 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.337392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:39017 (local address 127.24.83.1:45457)
0806 12:57:07.337546 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:07.337558 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:07.337636 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:07.337775 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:07.337776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:07.337777 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:07.337881 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:07.337883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.338749 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.338750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:07.339985 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:07.339988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.340710 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.340711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:07.340783 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.340789 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 12:57:07.340843 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 12:57:07.340898 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":7,"thread_start_us":69,"threads_started":1}
I20260806 12:57:07.341284 512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75150cc75c294fec8b8e205c00c6b720" candidate_uuid: "ca8117e7e3bc42949870a15d71e5b74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12b936d9c5c9436a8b1c01f2e870f359" is_pre_election: true
I20260806 12:57:07.341312 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75150cc75c294fec8b8e205c00c6b720" candidate_uuid: "ca8117e7e3bc42949870a15d71e5b74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52135d1b03a402b926b4d41be6962fa" is_pre_election: true
I20260806 12:57:07.341416 659 raft_consensus.cc:2454] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca8117e7e3bc42949870a15d71e5b74c in term 0.
I20260806 12:57:07.341419 512 raft_consensus.cc:2454] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca8117e7e3bc42949870a15d71e5b74c in term 0.
I20260806 12:57:07.341590 32763 leader_election.cc:302] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b936d9c5c9436a8b1c01f2e870f359, ca8117e7e3bc42949870a15d71e5b74c; no voters:
I20260806 12:57:07.341763 741 raft_consensus.cc:2781] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:57:07.341799 741 raft_consensus.cc:482] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:57:07.341813 741 raft_consensus.cc:3037] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.342474 741 raft_consensus.cc:504] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.342608 741 leader_election.cc:288] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [CANDIDATE]: Term 1 election: Requested vote from peers 12b936d9c5c9436a8b1c01f2e870f359 (127.24.83.2:42441), e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017)
I20260806 12:57:07.342749 512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75150cc75c294fec8b8e205c00c6b720" candidate_uuid: "ca8117e7e3bc42949870a15d71e5b74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12b936d9c5c9436a8b1c01f2e870f359"
I20260806 12:57:07.342772 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75150cc75c294fec8b8e205c00c6b720" candidate_uuid: "ca8117e7e3bc42949870a15d71e5b74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52135d1b03a402b926b4d41be6962fa"
I20260806 12:57:07.342839 659 raft_consensus.cc:3037] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.342839 512 raft_consensus.cc:3037] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.343461 659 raft_consensus.cc:2454] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca8117e7e3bc42949870a15d71e5b74c in term 1.
I20260806 12:57:07.343575 512 raft_consensus.cc:2454] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca8117e7e3bc42949870a15d71e5b74c in term 1.
I20260806 12:57:07.343640 32766 leader_election.cc:302] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca8117e7e3bc42949870a15d71e5b74c, e52135d1b03a402b926b4d41be6962fa; no voters:
I20260806 12:57:07.343727 741 raft_consensus.cc:2781] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:07.343875 741 raft_consensus.cc:686] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 1 LEADER]: Becoming Leader. State: Replica: ca8117e7e3bc42949870a15d71e5b74c, State: Running, Role: LEADER
I20260806 12:57:07.343983 741 consensus_queue.cc:237] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.344691 32671 catalog_manager.cc:5495] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c reported cstate change: term changed from 0 to 1, leader changed from <none> to ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1). New cstate: current_term: 1 leader_uuid: "ca8117e7e3bc42949870a15d71e5b74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } health_report { overall_health: UNKNOWN } } }
I20260806 12:57:07.352253 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.349879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56884 (local address 127.24.83.1:39747)
0806 12:57:07.349929 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:07.349931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:07.349943 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:57:07.350030 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:07.350030 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:07.350048 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:07.350108 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:57:07.350228 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.350703 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.351482 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.351683 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.351754 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.351931 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:57:07.352039 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:57:07.352122 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:07.352161 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 12:57:07.352180 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 12:57:07.353071 659 raft_consensus.cc:1261] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Refusing update from remote peer ca8117e7e3bc42949870a15d71e5b74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:07.353149 512 raft_consensus.cc:1261] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Refusing update from remote peer ca8117e7e3bc42949870a15d71e5b74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:07.353318 741 consensus_queue.cc:1035] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:07.353426 804 consensus_queue.cc:1035] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:07.356541 32671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:07.356638 32671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:07.357201 32671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f: Sending DeleteTablet for 3 replicas of tablet 75150cc75c294fec8b8e205c00c6b720
I20260806 12:57:07.357466 345 tablet_service.cc:1514] Processing DeleteTablet for tablet 75150cc75c294fec8b8e205c00c6b720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56880
I20260806 12:57:07.357501 639 tablet_service.cc:1514] Processing DeleteTablet for tablet 75150cc75c294fec8b8e205c00c6b720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:38842
I20260806 12:57:07.357573 760 tablet_replica.cc:330] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c: stopping tablet replica
I20260806 12:57:07.357561 491 tablet_service.cc:1514] Processing DeleteTablet for tablet 75150cc75c294fec8b8e205c00c6b720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:41162
I20260806 12:57:07.357590 758 tablet_replica.cc:330] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa: stopping tablet replica
I20260806 12:57:07.357621 760 raft_consensus.cc:2229] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:57:07.357635 758 raft_consensus.cc:2229] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:07.357636 759 tablet_replica.cc:330] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359: stopping tablet replica
I20260806 12:57:07.357679 759 raft_consensus.cc:2229] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:07.357697 760 raft_consensus.cc:2258] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.357698 758 raft_consensus.cc:2258] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.357713 759 raft_consensus.cc:2258] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.357998 760 ts_tablet_manager.cc:1905] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:07.358155 32671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:57:07.358285 32671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:57:07.358707 759 ts_tablet_manager.cc:1905] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:07.359544 760 ts_tablet_manager.cc:1918] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:07.359606 760 log.cc:1196] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-0/wal/wals/75150cc75c294fec8b8e205c00c6b720
I20260806 12:57:07.359807 760 ts_tablet_manager.cc:1939] T 75150cc75c294fec8b8e205c00c6b720 P ca8117e7e3bc42949870a15d71e5b74c: Deleting consensus metadata
I20260806 12:57:07.360262 759 ts_tablet_manager.cc:1918] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:07.360323 759 log.cc:1196] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/75150cc75c294fec8b8e205c00c6b720
I20260806 12:57:07.360525 759 ts_tablet_manager.cc:1939] T 75150cc75c294fec8b8e205c00c6b720 P 12b936d9c5c9436a8b1c01f2e870f359: Deleting consensus metadata
I20260806 12:57:07.360649 639 tablet_service.cc:1467] Processing CreateTablet for tablet cea55bf88b78403ab367696e3b405e3f (DEFAULT_TABLE table=test-table [id=e5779c0785c746d68dd02a9c5e5e502e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:07.360759 639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cea55bf88b78403ab367696e3b405e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.360821 32656 catalog_manager.cc:4841] TS 12b936d9c5c9436a8b1c01f2e870f359 (127.24.83.2:42441): tablet 75150cc75c294fec8b8e205c00c6b720 (table test-table [id=eb5f0d1914e745c88e8480b5abded72f]) successfully deleted
I20260806 12:57:07.360877 491 tablet_service.cc:1467] Processing CreateTablet for tablet cea55bf88b78403ab367696e3b405e3f (DEFAULT_TABLE table=test-table [id=e5779c0785c746d68dd02a9c5e5e502e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:07.360971 491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cea55bf88b78403ab367696e3b405e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.361696 735 tablet_bootstrap.cc:492] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359: Bootstrap starting.
I20260806 12:57:07.361737 345 tablet_service.cc:1467] Processing CreateTablet for tablet cea55bf88b78403ab367696e3b405e3f (DEFAULT_TABLE table=test-table [id=e5779c0785c746d68dd02a9c5e5e502e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:07.361824 345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cea55bf88b78403ab367696e3b405e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:07.362011 735 tablet_bootstrap.cc:654] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.362013 736 tablet_bootstrap.cc:492] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa: Bootstrap starting.
I20260806 12:57:07.362452 758 ts_tablet_manager.cc:1905] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:07.362684 736 tablet_bootstrap.cc:654] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.363293 736 tablet_bootstrap.cc:492] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa: No bootstrap required, opened a new log
I20260806 12:57:07.363353 736 ts_tablet_manager.cc:1397] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:07.363485 736 raft_consensus.cc:348] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.363543 736 raft_consensus.cc:374] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.363564 736 raft_consensus.cc:729] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e52135d1b03a402b926b4d41be6962fa, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.363616 736 consensus_queue.cc:260] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.363703 736 ts_tablet_manager.cc:1428] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:07.364045 32657 catalog_manager.cc:4841] TS ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747): tablet 75150cc75c294fec8b8e205c00c6b720 (table test-table [id=eb5f0d1914e745c88e8480b5abded72f]) successfully deleted
I20260806 12:57:07.364730 734 tablet_bootstrap.cc:492] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c: Bootstrap starting.
I20260806 12:57:07.363711 758 ts_tablet_manager.cc:1918] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:07.365058 758 log.cc:1196] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/75150cc75c294fec8b8e205c00c6b720
I20260806 12:57:07.365161 734 tablet_bootstrap.cc:654] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:07.365289 758 ts_tablet_manager.cc:1939] T 75150cc75c294fec8b8e205c00c6b720 P e52135d1b03a402b926b4d41be6962fa: Deleting consensus metadata
I20260806 12:57:07.365597 32659 catalog_manager.cc:4841] TS e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017): tablet 75150cc75c294fec8b8e205c00c6b720 (table test-table [id=eb5f0d1914e745c88e8480b5abded72f]) successfully deleted
I20260806 12:57:07.365856 735 tablet_bootstrap.cc:492] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359: No bootstrap required, opened a new log
I20260806 12:57:07.365908 735 ts_tablet_manager.cc:1397] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 12:57:07.366031 734 tablet_bootstrap.cc:492] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c: No bootstrap required, opened a new log
I20260806 12:57:07.366022 735 raft_consensus.cc:348] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.366084 734 ts_tablet_manager.cc:1397] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 12:57:07.366098 735 raft_consensus.cc:374] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.366128 735 raft_consensus.cc:729] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12b936d9c5c9436a8b1c01f2e870f359, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.366168 735 consensus_queue.cc:260] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.366215 734 raft_consensus.cc:348] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.366255 735 ts_tablet_manager.cc:1428] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:07.366281 734 raft_consensus.cc:374] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:07.366308 734 raft_consensus.cc:729] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8117e7e3bc42949870a15d71e5b74c, State: Initialized, Role: FOLLOWER
I20260806 12:57:07.366360 734 consensus_queue.cc:260] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.366463 734 ts_tablet_manager.cc:1428] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 12:57:07.587102 754 raft_consensus.cc:482] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:57:07.587218 754 raft_consensus.cc:504] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.587409 754 leader_election.cc:288] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747), e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017)
I20260806 12:57:07.587667 366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cea55bf88b78403ab367696e3b405e3f" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8117e7e3bc42949870a15d71e5b74c" is_pre_election: true
I20260806 12:57:07.587688 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cea55bf88b78403ab367696e3b405e3f" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52135d1b03a402b926b4d41be6962fa" is_pre_election: true
I20260806 12:57:07.587764 659 raft_consensus.cc:2454] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 0.
I20260806 12:57:07.587798 366 raft_consensus.cc:2454] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 0.
I20260806 12:57:07.587949 444 leader_election.cc:302] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b936d9c5c9436a8b1c01f2e870f359, e52135d1b03a402b926b4d41be6962fa; no voters:
I20260806 12:57:07.588073 754 raft_consensus.cc:2781] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:57:07.588129 754 raft_consensus.cc:482] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:57:07.588145 754 raft_consensus.cc:3037] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.588824 754 raft_consensus.cc:504] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.588959 754 leader_election.cc:288] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 election: Requested vote from peers ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747), e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017)
I20260806 12:57:07.589118 366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cea55bf88b78403ab367696e3b405e3f" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8117e7e3bc42949870a15d71e5b74c"
I20260806 12:57:07.589200 366 raft_consensus.cc:3037] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.589138 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cea55bf88b78403ab367696e3b405e3f" candidate_uuid: "12b936d9c5c9436a8b1c01f2e870f359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e52135d1b03a402b926b4d41be6962fa"
I20260806 12:57:07.589227 659 raft_consensus.cc:3037] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:07.589818 659 raft_consensus.cc:2454] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 1.
I20260806 12:57:07.589856 366 raft_consensus.cc:2454] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b936d9c5c9436a8b1c01f2e870f359 in term 1.
I20260806 12:57:07.590022 442 leader_election.cc:302] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b936d9c5c9436a8b1c01f2e870f359, ca8117e7e3bc42949870a15d71e5b74c; no voters:
I20260806 12:57:07.590119 754 raft_consensus.cc:2781] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:07.590169 754 raft_consensus.cc:686] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 LEADER]: Becoming Leader. State: Replica: 12b936d9c5c9436a8b1c01f2e870f359, State: Running, Role: LEADER
I20260806 12:57:07.590219 754 consensus_queue.cc:237] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } }
I20260806 12:57:07.590821 32671 catalog_manager.cc:5495] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12b936d9c5c9436a8b1c01f2e870f359 (127.24.83.2). New cstate: current_term: 1 leader_uuid: "12b936d9c5c9436a8b1c01f2e870f359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12b936d9c5c9436a8b1c01f2e870f359" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 42441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 } health_report { overall_health: UNKNOWN } } }
I20260806 12:57:07.651924 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.649676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41174 (local address 127.24.83.2:42441)
0806 12:57:07.649743 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:07.649745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:07.649752 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:57:07.649774 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:07.649774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:07.649791 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:07.649848 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:57:07.649974 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.650454 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.651198 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:07.651409 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:07.651487 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:07.651631 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:57:07.651666 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:57:07.651729 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:07.651797 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 12:57:07.651816 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 12:57:07.654031 659 raft_consensus.cc:1261] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Refusing update from remote peer 12b936d9c5c9436a8b1c01f2e870f359: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:07.654031 366 raft_consensus.cc:1261] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Refusing update from remote peer 12b936d9c5c9436a8b1c01f2e870f359: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:07.654246 740 consensus_queue.cc:1035] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e52135d1b03a402b926b4d41be6962fa" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 39017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:07.654361 745 consensus_queue.cc:1035] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca8117e7e3bc42949870a15d71e5b74c" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 39747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:07.663580 32670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:07.663688 32670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:07.664353 32670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75a391cae91a46f59ff1d95c3bb40e8f: Sending DeleteTablet for 3 replicas of tablet cea55bf88b78403ab367696e3b405e3f
I20260806 12:57:07.664573 345 tablet_service.cc:1514] Processing DeleteTablet for tablet cea55bf88b78403ab367696e3b405e3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56880
I20260806 12:57:07.664595 639 tablet_service.cc:1514] Processing DeleteTablet for tablet cea55bf88b78403ab367696e3b405e3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:38842
I20260806 12:57:07.664671 760 tablet_replica.cc:330] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c: stopping tablet replica
I20260806 12:57:07.664685 758 tablet_replica.cc:330] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa: stopping tablet replica
I20260806 12:57:07.664717 760 raft_consensus.cc:2229] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:07.664755 760 raft_consensus.cc:2258] T cea55bf88b78403ab367696e3b405e3f P ca8117e7e3bc42949870a15d71e5b74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.664757 758 raft_consensus.cc:2229] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:07.664796 758 raft_consensus.cc:2258] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.664937 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 32747
I20260806 12:57:07.665390 758 ts_tablet_manager.cc:1905] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:07.666333 491 tablet_service.cc:1514] Processing DeleteTablet for tablet cea55bf88b78403ab367696e3b405e3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:07 UTC) from {username='oryx', principal='oryx/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:41162
I20260806 12:57:07.666443 759 tablet_replica.cc:330] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359: stopping tablet replica
I20260806 12:57:07.666492 759 raft_consensus.cc:2229] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:57:07.666555 759 raft_consensus.cc:2258] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:07.666812 759 ts_tablet_manager.cc:1905] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:07.667258 758 ts_tablet_manager.cc:1918] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:57:07.667333 758 log.cc:1196] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/cea55bf88b78403ab367696e3b405e3f
I20260806 12:57:07.667562 758 ts_tablet_manager.cc:1939] T cea55bf88b78403ab367696e3b405e3f P e52135d1b03a402b926b4d41be6962fa: Deleting consensus metadata
I20260806 12:57:07.667919 32659 catalog_manager.cc:4841] TS e52135d1b03a402b926b4d41be6962fa (127.24.83.3:39017): tablet cea55bf88b78403ab367696e3b405e3f (table test-table [id=e5779c0785c746d68dd02a9c5e5e502e]) successfully deleted
I20260806 12:57:07.668201 759 ts_tablet_manager.cc:1918] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:57:07.668269 759 log.cc:1196] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/cea55bf88b78403ab367696e3b405e3f
I20260806 12:57:07.668469 759 ts_tablet_manager.cc:1939] T cea55bf88b78403ab367696e3b405e3f P 12b936d9c5c9436a8b1c01f2e870f359: Deleting consensus metadata
I20260806 12:57:07.668778 32656 catalog_manager.cc:4841] TS 12b936d9c5c9436a8b1c01f2e870f359 (127.24.83.2:42441): tablet cea55bf88b78403ab367696e3b405e3f (table test-table [id=e5779c0785c746d68dd02a9c5e5e502e]) successfully deleted
W20260806 12:57:07.673429 32657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:39747 (error 108)
I20260806 12:57:07.673583 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 423
I20260806 12:57:07.675252 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:07.675003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:39747 (local address 127.0.0.1:56886)
0806 12:57:07.675132 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:07.675200 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:39747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
W20260806 12:57:07.679679 32657 catalog_manager.cc:4584] TS ca8117e7e3bc42949870a15d71e5b74c (127.24.83.1:39747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cea55bf88b78403ab367696e3b405e3f: Network error: Client connection negotiation failed: client connection to 127.24.83.1:39747: connect: Connection refused (error 111)
I20260806 12:57:07.680035 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 570
I20260806 12:57:07.685698 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 32640
2026-08-06T12:57:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2968 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:57:07 dist-test-slave-zkpd krb5kdc[821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:57:07 dist-test-slave-zkpd krb5kdc[821](info): set up 2 sockets
Aug 06 12:57:07 dist-test-slave-zkpd krb5kdc[821](info): commencing operation
krb5kdc: starting...
W20260806 12:57:08.744534 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:57:08 dist-test-slave-zkpd krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:57:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:57:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:57:08.862385 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40387
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:40387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:08.942179 834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:08.942408 834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:08.942444 834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:08.943562 834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:57:08.943631 834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:08.943655 834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:08.943683 834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:57:08.943717 834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:57:08.945179 834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:40387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:08.945449 834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:08.945669 834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:08.950452 842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:08.950450 840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:08.950429 839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:08.950601 834 server_base.cc:1034] running on GCE node
I20260806 12:57:08.950824 834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:08.951115 834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:08.952275 834 hybrid_clock.cc:648] HybridClock initialized: now 1786021028952271 us; error 44 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T12:57:08Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1272 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:57:09 dist-test-slave-zkpd krb5kdc[847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:57:09 dist-test-slave-zkpd krb5kdc[847](info): set up 2 sockets
Aug 06 12:57:09 dist-test-slave-zkpd krb5kdc[847](info): commencing operation
krb5kdc: starting...
W20260806 12:57:10.000959 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:57:10 dist-test-slave-zkpd krb5kdc[847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:57:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:57:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:57:10.116000 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45843
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:36001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:45843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:10.196856 861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:10.197045 861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:10.197067 861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:10.198185 861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:57:10.198235 861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:10.198249 861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:10.198262 861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:57:10.198274 861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:57:10.199766 861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:45843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:10.200043 861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:10.200268 861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:10.204882 868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:10.204924 867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:10.205062 861 server_base.cc:1034] running on GCE node
W20260806 12:57:10.204890 870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:10.205406 861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:10.205693 861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:10.206843 861 hybrid_clock.cc:648] HybridClock initialized: now 1786021030206828 us; error 27 us; skew 500 ppm
Aug 06 12:57:10 dist-test-slave-zkpd krb5kdc[847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021030, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:10.209261 861 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:57:10.209687 861 webserver.cc:460] Webserver started at http://127.24.83.62:38355/ using document root <none> and password file <none>
I20260806 12:57:10.209918 861 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:10.209982 861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:10.210125 861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:10.211115 861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "12e2b1fd771447ecacc69654ffcded79"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "6ebb26af3f36e38c1f9426466f7ba1d3"
server_key_iv: "d50e7b89a2dfd486d1b5fccd40bed1e1"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.211470 861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "12e2b1fd771447ecacc69654ffcded79"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "6ebb26af3f36e38c1f9426466f7ba1d3"
server_key_iv: "d50e7b89a2dfd486d1b5fccd40bed1e1"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.212860 861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:57:10.213588 877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:10.213806 861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:57:10.213912 861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "12e2b1fd771447ecacc69654ffcded79"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "6ebb26af3f36e38c1f9426466f7ba1d3"
server_key_iv: "d50e7b89a2dfd486d1b5fccd40bed1e1"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.213984 861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:10.228538 861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:10.230355 861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:10.230515 861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:10.234571 861 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:45843
I20260806 12:57:10.234619 930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:45843 every 8 connection(s)
I20260806 12:57:10.234969 861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:57:10.235777 931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:10.237916 931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79: Bootstrap starting.
I20260806 12:57:10.238406 931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:10.238662 931 log.cc:824] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:10.239287 931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79: No bootstrap required, opened a new log
I20260806 12:57:10.240401 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 861
I20260806 12:57:10.240521 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:57:10.240729 24908 external_mini_cluster.cc:1428] Setting key 44910c85151cc9a635be0c6c45518bf9
I20260806 12:57:10.241303 931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } }
I20260806 12:57:10.241485 931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:10.241513 931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e2b1fd771447ecacc69654ffcded79, State: Initialized, Role: FOLLOWER
I20260806 12:57:10.241650 931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } }
I20260806 12:57:10.241719 931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:57:10.241760 931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:57:10.241844 931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:10.242560 931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } }
I20260806 12:57:10.242657 931 leader_election.cc:302] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12e2b1fd771447ecacc69654ffcded79; no voters:
I20260806 12:57:10.242828 931 leader_election.cc:288] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:57:10.242862 936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:10.243001 936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 1 LEADER]: Becoming Leader. State: Replica: 12e2b1fd771447ecacc69654ffcded79, State: Running, Role: LEADER
I20260806 12:57:10.243086 931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:57:10.243170 936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } }
I20260806 12:57:10.243537 936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12e2b1fd771447ecacc69654ffcded79. Latest consensus state: current_term: 1 leader_uuid: "12e2b1fd771447ecacc69654ffcded79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } } }
I20260806 12:57:10.243634 938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12e2b1fd771447ecacc69654ffcded79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } } }
I20260806 12:57:10.243669 936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:10.243713 938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:10.243898 943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:57:10.244725 943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:57:10.246212 943 catalog_manager.cc:1347] Generated new cluster ID: a7ac91f6f2c84f6fa565792f78c9efbe
I20260806 12:57:10.246261 943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:57:10 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:57:10.251430 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:10.241227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49538 (local address 127.24.83.62:45843)
0806 12:57:10.241407 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:10.241427 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:10.241450 (+ 23us) server_negotiation.cc:407] Connection header received
0806 12:57:10.241586 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:10.241592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:10.241644 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:10.241799 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:10.242523 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.243777 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.245130 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.245663 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.245766 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:10.248070 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:10.248074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:10.248075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:10.248086 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:10.249783 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:10.250146 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:10.250150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:10.250150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:10.250249 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:10.250469 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:10.250470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:10.250471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:10.250689 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:10.250845 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:10.250953 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 12:57:10.251289 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260806 12:57:10.271195 943 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:57:10.271708 943 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:57:10.282459 943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79: Generated new TSK 0
I20260806 12:57:10.282671 943 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:57:10.313000 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45843
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:36001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:10.409068 957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:10.409261 957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:10.409286 957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:10.410387 957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:10.410420 957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:10.410445 957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:57:10.411785 957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:10.412020 957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:10.412246 957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:10.417274 965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:10.417285 963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:10.417282 962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:10.417461 957 server_base.cc:1034] running on GCE node
I20260806 12:57:10.417654 957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:10.417881 957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:10.419031 957 hybrid_clock.cc:648] HybridClock initialized: now 1786021030419022 us; error 26 us; skew 500 ppm
Aug 06 12:57:10 dist-test-slave-zkpd krb5kdc[847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021030, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:10.421499 957 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:57:10.421854 957 webserver.cc:460] Webserver started at http://127.24.83.1:46497/ using document root <none> and password file <none>
I20260806 12:57:10.422034 957 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:10.422075 957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:10.422154 957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:10.423067 957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "7c0e929903ce428e9f29dbc69aa58086"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "69c05676d85cc01d8818f036d5baf21e"
server_key_iv: "b8227397fabc7da5a7f67916c340a5f4"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.423367 957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "7c0e929903ce428e9f29dbc69aa58086"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "69c05676d85cc01d8818f036d5baf21e"
server_key_iv: "b8227397fabc7da5a7f67916c340a5f4"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.424676 957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:57:10.425371 972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:10.425590 957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:57:10.425673 957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "7c0e929903ce428e9f29dbc69aa58086"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "69c05676d85cc01d8818f036d5baf21e"
server_key_iv: "b8227397fabc7da5a7f67916c340a5f4"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.425727 957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:10.440261 957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:10.442116 957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:10.442224 957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:10.442412 957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:10.442641 957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:10.442684 957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:10.442705 957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:10.442759 957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:10.448920 957 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:38751
I20260806 12:57:10.449350 957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:57:10.454517 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 957
I20260806 12:57:10.454623 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:57:10.454830 24908 external_mini_cluster.cc:1428] Setting key 43ea7c5cf276ea37a232da1cff90d834
I20260806 12:57:10.455341 1087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:38751 every 8 connection(s)
Aug 06 12:57:10 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021030, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
I20260806 12:57:10.464874 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:10.455603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:39135 (local address 127.24.83.62:45843)
0806 12:57:10.455651 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:10.455653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:10.455726 (+ 73us) server_negotiation.cc:407] Connection header received
0806 12:57:10.456331 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:10.456332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:10.456349 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:10.456426 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:10.457418 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.457914 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.459071 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.459425 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.459508 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:10.461908 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:10.461910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:10.461910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:10.461919 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:10.463541 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:10.463923 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:10.463924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:10.463925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:10.463989 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:10.464286 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:10.464287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:10.464287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:10.464437 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:10.464512 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:10.464730 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 12:57:10.464812 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 12:57:10.465003 1090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:10.455294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:39135)
0806 12:57:10.455602 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:10.455641 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:10.456236 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:10.456475 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:10.456478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:10.457026 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:10.457314 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:10.457321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.458045 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.458046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:10.458981 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:10.458983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.459517 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.459518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:10.459676 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:10.460223 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:10.460232 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:10.461809 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:10.463624 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:10.463630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:10.463631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:10.463831 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:10.464061 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:10.464061 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:10.464062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:10.464229 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:10.464506 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:10.464510 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:10.464616 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 12:57:10.464695 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 12:57:10.464806 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":83,"threads_started":1}
I20260806 12:57:10.465514 1088 heartbeater.cc:347] Connected to a master server at 127.24.83.62:45843
I20260806 12:57:10.465606 1088 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:10.465791 1088 heartbeater.cc:511] Master 127.24.83.62:45843 requested a full tablet report, sending...
I20260806 12:57:10.466389 895 ts_manager.cc:194] Registered new tserver with Master: 7c0e929903ce428e9f29dbc69aa58086 (127.24.83.1:38751)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
I20260806 12:57:10.467263 895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:39135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:57:10.481950 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45843
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:36001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:10.561151 1093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:10.561331 1093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:10.561352 1093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:10.562412 1093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:10.562444 1093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:10.562469 1093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:57:10.563768 1093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:10.564011 1093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:10.564188 1093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:10.568890 1098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:10.568949 1099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:10.569097 1093 server_base.cc:1034] running on GCE node
W20260806 12:57:10.568890 1101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:10.569379 1093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:10.569566 1093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:10.570726 1093 hybrid_clock.cc:648] HybridClock initialized: now 1786021030570706 us; error 31 us; skew 500 ppm
Aug 06 12:57:10 dist-test-slave-zkpd krb5kdc[847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021030, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:10.573179 1093 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:57:10.573630 1093 webserver.cc:460] Webserver started at http://127.24.83.2:36571/ using document root <none> and password file <none>
I20260806 12:57:10.573881 1093 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:10.573953 1093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:10.574098 1093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:10.575110 1093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "b4193c61d0004e5c8832b899ebea47e1"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "ef2745388ac66e6b43c098012e012977"
server_key_iv: "a36b4e0a45b87b22ce9b6c3a719eae31"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.575469 1093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "b4193c61d0004e5c8832b899ebea47e1"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "ef2745388ac66e6b43c098012e012977"
server_key_iv: "a36b4e0a45b87b22ce9b6c3a719eae31"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.576830 1093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:57:10.577648 1108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:10.577823 1093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 12:57:10.577921 1093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "b4193c61d0004e5c8832b899ebea47e1"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "ef2745388ac66e6b43c098012e012977"
server_key_iv: "a36b4e0a45b87b22ce9b6c3a719eae31"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.578006 1093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:10.595513 1093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:10.597337 1093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:10.597498 1093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:10.597714 1093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:10.598016 1093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:10.598069 1093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:10.598104 1093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:10.598130 1093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:10.603956 1093 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:44493
I20260806 12:57:10.603989 1223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:44493 every 8 connection(s)
I20260806 12:57:10.604295 1093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:57:10.606365 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 1093
I20260806 12:57:10.606490 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:57:10.606696 24908 external_mini_cluster.cc:1428] Setting key c50d6f12a0ec444169eab22b042b035d
Aug 06 12:57:10 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021030, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:57:10.614317 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:10.604999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:54073 (local address 127.24.83.62:45843)
0806 12:57:10.605090 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:10.605093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:10.605283 (+ 190us) server_negotiation.cc:407] Connection header received
0806 12:57:10.605783 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:10.605785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:10.605798 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:10.605857 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:10.607035 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.607671 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.608725 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.608933 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.609010 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:10.611391 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:10.611392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:10.611392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:10.611399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:10.612835 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:10.613306 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:10.613306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:10.613307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:10.613356 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:10.613635 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:10.613636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:10.613636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:10.613783 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:10.613855 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:10.614175 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 12:57:10.614245 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 12:57:10.614562 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:10.604927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:54073)
0806 12:57:10.605141 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:10.605199 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:10.605679 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:10.605936 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:10.605939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:10.606670 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:10.606940 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:10.606945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.607770 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.607771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:10.608624 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:10.608626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.609017 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.609018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:10.609147 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:10.609626 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:10.609634 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:10.611300 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:10.612915 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:10.612922 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:10.612925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:10.613224 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:10.613425 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:10.613426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:10.613426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:10.613563 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:10.613860 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:10.613867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:10.614032 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 12:57:10.614154 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 12:57:10.614320 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260806 12:57:10.615092 1224 heartbeater.cc:347] Connected to a master server at 127.24.83.62:45843
I20260806 12:57:10.615192 1224 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:10.615370 1224 heartbeater.cc:511] Master 127.24.83.62:45843 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
I20260806 12:57:10.615765 895 ts_manager.cc:194] Registered new tserver with Master: b4193c61d0004e5c8832b899ebea47e1 (127.24.83.2:44493)
I20260806 12:57:10.616185 895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:54073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:57:10.632193 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45843
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:36001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:10.711540 1229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:10.711779 1229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:10.711817 1229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:10.712980 1229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:10.713048 1229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:10.713093 1229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:57:10.714545 1229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:10.714803 1229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:10.715045 1229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:10.719964 1234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:10.720070 1229 server_base.cc:1034] running on GCE node
W20260806 12:57:10.720178 1237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:10.720012 1235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:10.720449 1229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:10.720716 1229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:10.721854 1229 hybrid_clock.cc:648] HybridClock initialized: now 1786021030721832 us; error 35 us; skew 500 ppm
Aug 06 12:57:10 dist-test-slave-zkpd krb5kdc[847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021030, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:10.724313 1229 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:57:10.724716 1229 webserver.cc:460] Webserver started at http://127.24.83.3:44371/ using document root <none> and password file <none>
I20260806 12:57:10.724960 1229 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:10.725035 1229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:10.725188 1229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:10.726220 1229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "bd36bb70aced4540a1d1ae8dd39cd857"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "3fede76630e3311fed70457c27748b77"
server_key_iv: "fc1e77a495cfa24dd26720d62ab53ab7"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.726583 1229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "bd36bb70aced4540a1d1ae8dd39cd857"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "3fede76630e3311fed70457c27748b77"
server_key_iv: "fc1e77a495cfa24dd26720d62ab53ab7"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.727835 1229 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:57:10.728646 1244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:10.728824 1229 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:10.728922 1229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "bd36bb70aced4540a1d1ae8dd39cd857"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "3fede76630e3311fed70457c27748b77"
server_key_iv: "fc1e77a495cfa24dd26720d62ab53ab7"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.729007 1229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:10.743357 1229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:10.745752 1229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:10.745915 1229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:10.746140 1229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:10.746438 1229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:10.746496 1229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:10.746538 1229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:10.746567 1229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:10.752285 1229 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:35145
I20260806 12:57:10.752385 1361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:35145 every 8 connection(s)
I20260806 12:57:10.752672 1229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:57:10.756300 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 1229
I20260806 12:57:10.756383 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:57:10.756594 24908 external_mini_cluster.cc:1428] Setting key 15c7cd4c1ac91b35c75a6f560d5ea15d
Aug 06 12:57:10 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021030, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:57:10.761646 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:10.753356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:32897 (local address 127.24.83.62:45843)
0806 12:57:10.753438 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:10.753440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:10.753634 (+ 194us) server_negotiation.cc:407] Connection header received
0806 12:57:10.754087 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:10.754087 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:10.754097 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:10.754149 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:10.754976 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.755453 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.756813 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.757110 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.757194 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:10.759165 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:10.759167 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:10.759167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:10.759173 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:10.760565 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:10.760896 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:10.760897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:10.760897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:10.760947 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:10.761154 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:10.761155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:10.761155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:10.761249 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:10.761305 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:10.761523 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 12:57:10.761581 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 12:57:10.761792 1364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:10.753326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:32897)
0806 12:57:10.753518 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:10.753552 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:10.754009 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:10.754200 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:10.754203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:10.754728 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:10.754906 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:10.754910 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.755530 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.755530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:10.756728 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:10.756731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:10.757221 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:10.757222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:10.757363 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:10.757801 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:10.757808 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:10.759086 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:10.760632 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:10.760637 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:10.760638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:10.760833 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:10.761011 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:10.761012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:10.761012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:10.761100 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:10.761301 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:10.761304 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:10.761417 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 12:57:10.761494 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 12:57:10.761604 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":103,"threads_started":1}
I20260806 12:57:10.762253 1362 heartbeater.cc:347] Connected to a master server at 127.24.83.62:45843
I20260806 12:57:10.762359 1362 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:10.762532 1362 heartbeater.cc:511] Master 127.24.83.62:45843 requested a full tablet report, sending...
I20260806 12:57:10.762916 895 ts_manager.cc:194] Registered new tserver with Master: bd36bb70aced4540a1d1ae8dd39cd857 (127.24.83.3:35145)
I20260806 12:57:10.763276 895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:32897
I20260806 12:57:10.768217 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab.
Entry for principal oryx/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab.
WARNING: no policy specified for oryx/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab.
Entry for principal oryx/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab.
WARNING: no policy specified for oryx/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab.
Entry for principal oryx/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab.
I20260806 12:57:10.835577 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 957
I20260806 12:57:10.840818 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 1093
I20260806 12:57:10.846663 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 1229
I20260806 12:57:10.851146 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 861
I20260806 12:57:10.856895 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45843
--webserver_interface=127.24.83.62
--webserver_port=38355
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:36001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:45843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:10.938279 1372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:10.938480 1372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:10.938506 1372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:10.939679 1372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:57:10.939723 1372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:10.939736 1372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:10.939749 1372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:57:10.939800 1372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:57:10.941310 1372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:45843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=38355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:10.941576 1372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:10.941805 1372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:10.946532 1377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:10.946522 1378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:10.946593 1372 server_base.cc:1034] running on GCE node
W20260806 12:57:10.946522 1380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:10.946944 1372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:10.947253 1372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:10.948395 1372 hybrid_clock.cc:648] HybridClock initialized: now 1786021030948375 us; error 35 us; skew 500 ppm
Aug 06 12:57:10 dist-test-slave-zkpd krb5kdc[847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021030, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:10.950764 1372 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:57:10.951218 1372 webserver.cc:460] Webserver started at http://127.24.83.62:38355/ using document root <none> and password file <none>
I20260806 12:57:10.951459 1372 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:10.951516 1372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:10.952903 1372 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:10.953519 1388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:10.953703 1372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:10.953821 1372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "12e2b1fd771447ecacc69654ffcded79"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "6ebb26af3f36e38c1f9426466f7ba1d3"
server_key_iv: "d50e7b89a2dfd486d1b5fccd40bed1e1"
server_key_version: "encryptionkey@0"
I20260806 12:57:10.954137 1372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:10.985309 1372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:10.987164 1372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:10.987322 1372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:10.991353 1372 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:45843
I20260806 12:57:10.991381 1440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:45843 every 8 connection(s)
I20260806 12:57:10.991762 1372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:57:10.993181 1441 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:57:10.993790 1441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79: Bootstrap starting.
I20260806 12:57:11.001992 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 1372
I20260806 12:57:11.001977 1441 log.cc:824] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:11.002367 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:38751
--local_ip_for_outbound_sockets=127.24.83.1
--tserver_master_addrs=127.24.83.62:45843
--webserver_port=46497
--webserver_interface=127.24.83.1
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:36001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.83.1
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:57:11.004236 1441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:57:11.004530 1441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79: Bootstrap complete.
I20260806 12:57:11.006893 1441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } }
I20260806 12:57:11.007108 1441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e2b1fd771447ecacc69654ffcded79, State: Initialized, Role: FOLLOWER
I20260806 12:57:11.007267 1441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } }
I20260806 12:57:11.007336 1441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:57:11.007403 1441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:57:11.007450 1441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:57:11.008503 1441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } }
I20260806 12:57:11.008590 1441 leader_election.cc:302] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12e2b1fd771447ecacc69654ffcded79; no voters:
I20260806 12:57:11.008816 1441 leader_election.cc:288] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 12:57:11.008849 1445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 12:57:11.008971 1445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [term 2 LEADER]: Becoming Leader. State: Replica: 12e2b1fd771447ecacc69654ffcded79, State: Running, Role: LEADER
I20260806 12:57:11.009085 1441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:57:11.009126 1445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } }
I20260806 12:57:11.009419 1445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12e2b1fd771447ecacc69654ffcded79. Latest consensus state: current_term: 2 leader_uuid: "12e2b1fd771447ecacc69654ffcded79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } } }
I20260806 12:57:11.009498 1445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:11.009423 1446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "12e2b1fd771447ecacc69654ffcded79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e2b1fd771447ecacc69654ffcded79" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45843 } } }
I20260806 12:57:11.009801 1446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:11.009743 1451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:57:11.010476 1451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:57:11.010866 1451 catalog_manager.cc:1259] Loaded cluster ID: a7ac91f6f2c84f6fa565792f78c9efbe
I20260806 12:57:11.010959 1451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:57:11.012385 1451 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:57:11.012586 1451 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 12e2b1fd771447ecacc69654ffcded79: Loaded TSK: 0
I20260806 12:57:11.012850 1451 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 12:57:11.084424 1444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:11.084635 1444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:11.084659 1444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:11.086181 1444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:11.086233 1444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:11.086261 1444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:57:11.087836 1444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.1.keytab
--principal=oryx/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:38751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=46497
--tserver_master_addrs=127.24.83.62:45843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:11.088130 1444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:11.088361 1444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:11.094153 1469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:11.094153 1471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:11.094266 1468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:11.094488 1444 server_base.cc:1034] running on GCE node
I20260806 12:57:11.094703 1444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:11.094997 1444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:11.096143 1444 hybrid_clock.cc:648] HybridClock initialized: now 1786021031096133 us; error 27 us; skew 500 ppm
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021031, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:11.098874 1444 init.cc:377] Logged in from keytab as oryx/127.24.83.1@KRBTEST.COM (short username oryx)
I20260806 12:57:11.099404 1444 webserver.cc:460] Webserver started at http://127.24.83.1:46497/ using document root <none> and password file <none>
I20260806 12:57:11.099644 1444 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:11.099716 1444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:11.101503 1444 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:11.102263 1478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:11.102474 1444 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:11.102571 1444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "7c0e929903ce428e9f29dbc69aa58086"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "69c05676d85cc01d8818f036d5baf21e"
server_key_iv: "b8227397fabc7da5a7f67916c340a5f4"
server_key_version: "encryptionkey@0"
I20260806 12:57:11.102923 1444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:11.116585 1444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:11.118902 1444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:11.119071 1444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:11.119300 1444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:11.119621 1444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:11.119676 1444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:11.119719 1444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:11.119745 1444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:11.125778 1444 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:38751
I20260806 12:57:11.125841 1592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:38751 every 8 connection(s)
I20260806 12:57:11.126168 1444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:57:11.127609 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 1444
I20260806 12:57:11.127954 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:44493
--local_ip_for_outbound_sockets=127.24.83.2
--tserver_master_addrs=127.24.83.62:45843
--webserver_port=36571
--webserver_interface=127.24.83.2
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:36001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.83.2
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:11.135067 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:11.126866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:33727)
0806 12:57:11.127060 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:11.127101 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:11.127668 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:11.128335 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:11.128339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:11.129162 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:11.129449 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.129457 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.130628 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.130629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.131452 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.131454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.132132 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.132133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.132235 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.132740 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:11.132748 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:11.134864 (+ 2116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
W20260806 12:57:11.135138 1596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:11.126969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:33727 (local address 127.24.83.62:45843)
0806 12:57:11.127858 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:11.127866 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:11.127933 (+ 67us) server_negotiation.cc:407] Connection header received
0806 12:57:11.128057 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:11.128063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:11.128112 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:11.128263 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:11.129603 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.130510 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.131556 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.132004 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.132477 (+ 473us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.134979 (+ 2502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:33727: BlockingRecv error: recv got EOF from 127.24.83.1:33727 (error 108)
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":74,"threads_started":1}
W20260806 12:57:11.135632 1593 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:45843 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.83.62:45843: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:11.139953 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:11.135888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:33805)
0806 12:57:11.135963 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:11.135973 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:11.136014 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:11.136170 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:11.136171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:11.136300 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:11.136345 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.136346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.136948 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.136949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.137439 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.137441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.137806 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.137807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.137877 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.138278 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:11.138282 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:11.139865 (+ 1583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 12:57:11.140029 1596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:11.135952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:33805 (local address 127.24.83.62:45843)
0806 12:57:11.135995 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:11.135997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:11.136005 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:11.136073 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:11.136074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:11.136085 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:11.136154 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:11.136423 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.136869 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.137536 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.137719 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.137801 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.139968 (+ 2167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:33805: BlockingRecv error: recv got EOF from 127.24.83.1:33805 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:11.144294 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:11.140409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:55751)
0806 12:57:11.140447 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:11.140455 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:11.140510 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:11.140622 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:11.140623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:11.140739 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:11.140762 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.140763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.141377 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.141378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.141852 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.141854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.142236 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.142237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.142287 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.142678 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:11.142683 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:11.144218 (+ 1535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 12:57:11.144346 1596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:11.140462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:55751 (local address 127.24.83.62:45843)
0806 12:57:11.140521 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:11.140523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:11.140531 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:11.140553 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:11.140554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:11.140563 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:11.140615 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:11.140840 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.141281 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.141929 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.142107 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.142181 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.144282 (+ 2101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:55751: BlockingRecv error: recv got EOF from 127.24.83.1:55751 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 12:57:11.144625 1593 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:57:11.214082 1597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:11.214272 1597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:11.214295 1597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:11.215387 1597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:11.215425 1597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:11.215449 1597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:57:11.216806 1597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.2.keytab
--principal=oryx/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:44493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=36571
--tserver_master_addrs=127.24.83.62:45843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:11.216969 1597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:11.217125 1597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:57:11.221956 1597 server_base.cc:1034] running on GCE node
W20260806 12:57:11.221896 1603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:11.221916 1602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:11.221896 1605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:11.222357 1597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:11.222644 1597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:11.223821 1597 hybrid_clock.cc:648] HybridClock initialized: now 1786021031223807 us; error 26 us; skew 500 ppm
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021031, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:11.226167 1597 init.cc:377] Logged in from keytab as oryx/127.24.83.2@KRBTEST.COM (short username oryx)
I20260806 12:57:11.226639 1597 webserver.cc:460] Webserver started at http://127.24.83.2:36571/ using document root <none> and password file <none>
I20260806 12:57:11.226850 1597 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:11.226960 1597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:11.228286 1597 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:11.228827 1612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:11.229072 1597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:11.229216 1597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "b4193c61d0004e5c8832b899ebea47e1"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "ef2745388ac66e6b43c098012e012977"
server_key_iv: "a36b4e0a45b87b22ce9b6c3a719eae31"
server_key_version: "encryptionkey@0"
I20260806 12:57:11.229550 1597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:11.244139 1597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:11.245898 1597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:11.246052 1597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:11.246265 1597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:11.246554 1597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:11.246608 1597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:11.246647 1597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:11.246675 1597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:11.252663 1597 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:44493
I20260806 12:57:11.252708 1726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:44493 every 8 connection(s)
I20260806 12:57:11.253019 1597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:57:11.253913 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 1597
I20260806 12:57:11.254242 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:35145
--local_ip_for_outbound_sockets=127.24.83.3
--tserver_master_addrs=127.24.83.62:45843
--webserver_port=44371
--webserver_interface=127.24.83.3
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:36001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.83.3
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:11.261368 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:11.253645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:50871)
0806 12:57:11.253872 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:11.253920 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:11.254631 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:11.254889 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:11.254893 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:11.255702 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:11.256018 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.256023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.257042 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.257043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.257903 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.257906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.258391 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.258392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.258486 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.258984 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:11.258991 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:11.261227 (+ 2236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":124,"threads_started":1}
W20260806 12:57:11.261449 1596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:11.253652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:50871 (local address 127.24.83.62:45843)
0806 12:57:11.253712 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:11.253715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:11.254022 (+ 307us) server_negotiation.cc:407] Connection header received
0806 12:57:11.254739 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:11.254740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:11.254754 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:11.254812 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:11.256111 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.256936 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.257968 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.258301 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.258379 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.261360 (+ 2981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:50871: BlockingRecv error: recv got EOF from 127.24.83.2:50871 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 12:57:11.261605 1727 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:45843 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.83.62:45843: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:11.265972 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:11.261854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:60465)
0806 12:57:11.261892 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:11.261902 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:11.261959 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:11.262089 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:11.262090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:11.262212 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:11.262250 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.262251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.262860 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.262861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.263362 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.263363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.263724 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.263725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.263793 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.264231 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:11.264235 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:11.265896 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 12:57:11.266026 1596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:11.261919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:60465 (local address 127.24.83.62:45843)
0806 12:57:11.261984 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:11.261986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:11.261996 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:57:11.262019 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:11.262020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:11.262029 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:11.262085 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:11.262324 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.262777 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.263454 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.263643 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.263709 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.265964 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:60465: BlockingRecv error: recv got EOF from 127.24.83.2:60465 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:11.270798 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:11.266210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:36185)
0806 12:57:11.266350 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:11.266359 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:11.266414 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:11.266531 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:11.266531 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:11.266663 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:11.266704 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.266705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.267564 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.267565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.268156 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.268157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.268641 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.268642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.268684 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.269101 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:11.269104 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:11.270729 (+ 1625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
W20260806 12:57:11.270895 1596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:11.266262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:36185 (local address 127.24.83.62:45843)
0806 12:57:11.266323 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:11.266325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:11.266421 (+ 96us) server_negotiation.cc:407] Connection header received
0806 12:57:11.266450 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:11.266451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:11.266460 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:11.266521 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:11.266765 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.267491 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.268267 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.268558 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.268618 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.270840 (+ 2222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:36185: BlockingRecv error: recv got EOF from 127.24.83.2:36185 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 12:57:11.270915 1727 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:57:11.340551 1731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:11.340797 1731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:11.340837 1731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:11.342016 1731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:11.342085 1731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:11.342131 1731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:57:11.343688 1731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/krb5kdc/oryx_127.24.83.3.keytab
--principal=oryx/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:35145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=44371
--tserver_master_addrs=127.24.83.62:45843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:11.343984 1731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:11.344214 1731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:11.349066 1737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:11.349089 1731 server_base.cc:1034] running on GCE node
W20260806 12:57:11.349176 1738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:11.349089 1740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:11.349465 1731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:11.349736 1731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:11.350906 1731 hybrid_clock.cc:648] HybridClock initialized: now 1786021031350881 us; error 40 us; skew 500 ppm
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021031, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:11.353310 1731 init.cc:377] Logged in from keytab as oryx/127.24.83.3@KRBTEST.COM (short username oryx)
I20260806 12:57:11.353732 1731 webserver.cc:460] Webserver started at http://127.24.83.3:44371/ using document root <none> and password file <none>
I20260806 12:57:11.353966 1731 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:11.354049 1731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:11.355368 1731 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:57:11.355927 1747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:11.356104 1731 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:57:11.356202 1731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "bd36bb70aced4540a1d1ae8dd39cd857"
format_stamp: "Formatted at 2026-08-06 12:57:10 on dist-test-slave-zkpd"
server_key: "3fede76630e3311fed70457c27748b77"
server_key_iv: "fc1e77a495cfa24dd26720d62ab53ab7"
server_key_version: "encryptionkey@0"
I20260806 12:57:11.356542 1731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:11.380277 1731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:11.382117 1731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:11.382268 1731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:11.382508 1731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:11.382807 1731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:11.382879 1731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:11.382947 1731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:11.382975 1731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:11.388460 1731 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:35145
I20260806 12:57:11.388504 1861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:35145 every 8 connection(s)
I20260806 12:57:11.388825 1731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:57:11.389758 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 1731
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 12:57:11.397068 1596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:11.389472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:34785 (local address 127.24.83.62:45843)
0806 12:57:11.389532 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:11.389534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:11.389728 (+ 194us) server_negotiation.cc:407] Connection header received
0806 12:57:11.390468 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:11.390468 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:11.390479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:11.390530 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:11.391640 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.392262 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.393555 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.393741 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.393816 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.397023 (+ 3207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:34785: BlockingRecv error: recv got EOF from 127.24.83.3:34785 (error 108)
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":35}
I20260806 12:57:11.397193 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:11.389433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:34785)
0806 12:57:11.389592 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:11.389632 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:11.390366 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:11.390614 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:11.390619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:11.391364 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:11.391557 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.391563 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.392339 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.392340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.393479 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.393482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.393854 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.393855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.394027 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.394441 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:11.394452 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:11.396981 (+ 2529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
W20260806 12:57:11.397495 1862 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:45843 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.83.62:45843: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
I20260806 12:57:11.398461 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:11.390296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46174 (local address 127.24.83.62:45843)
0806 12:57:11.390422 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:11.390424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:11.390436 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:57:11.390468 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:11.390468 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:11.390479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:11.390536 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:11.391355 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.392058 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.393102 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.393307 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.393369 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.394906 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:11.394910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:11.394911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:11.394921 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:11.396807 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:11.397215 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:11.397218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:11.397219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:11.397346 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:11.397586 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:11.397587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:11.397587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:11.397943 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:11.398090 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:11.398211 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 12:57:11.398343 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:11.401891 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:11.397751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:51309)
0806 12:57:11.397794 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:11.397804 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:11.397836 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:11.397956 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:11.397957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:11.398089 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:11.398158 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.398160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.399032 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.399033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.399670 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.399673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.400034 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.400034 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.400077 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.400376 (+ 299us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:11.400379 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:11.401852 (+ 1473us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 12:57:11.401983 1596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:11.397786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:51309 (local address 127.24.83.62:45843)
0806 12:57:11.397819 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:11.397821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:11.397829 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:11.397886 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:11.397886 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:11.397907 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:11.397950 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:11.398243 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.398919 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.399740 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.399974 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.400033 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.401929 (+ 1896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:51309: BlockingRecv error: recv got EOF from 127.24.83.3:51309 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:11 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:11.405480 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:11.402175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:44851)
0806 12:57:11.402260 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:11.402269 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:11.402308 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:11.402398 (+ 90us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:11.402398 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:11.402500 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:11.402521 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.402522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.403002 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.403002 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.403439 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:11.403440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.403714 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.403715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:11.403746 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.404073 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:11.404077 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:11.405441 (+ 1364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 12:57:11.405579 1596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:11.402227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:44851 (local address 127.24.83.62:45843)
0806 12:57:11.402264 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:11.402265 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:11.402297 (+ 32us) server_negotiation.cc:407] Connection header received
0806 12:57:11.402342 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:11.402342 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:11.402349 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:11.402388 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:11.402554 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.402930 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.403502 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:11.403657 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:11.403702 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:11.405515 (+ 1813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:44851: BlockingRecv error: recv got EOF from 127.24.83.3:44851 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 12:57:11.405637 1862 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 12:57:12 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:12 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:12.149515 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:12.145014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:39609)
0806 12:57:12.145147 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:12.145163 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:12.145239 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:12.145362 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:12.145363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:12.145517 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:12.145569 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:12.145571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.146228 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.146229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:12.146783 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:12.146784 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.147119 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.147120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:12.147160 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:12.147603 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:12.147607 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:12.149437 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
W20260806 12:57:12.149648 1868 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:12.145050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:39609 (local address 127.24.83.62:45843)
0806 12:57:12.145172 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:12.145174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:12.145222 (+ 48us) server_negotiation.cc:407] Connection header received
0806 12:57:12.145280 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:12.145281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:12.145290 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:12.145347 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:12.145631 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.146147 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.146862 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.147057 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.147106 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:12.149541 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:39609: BlockingRecv error: recv got EOF from 127.24.83.1:39609 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
Aug 06 12:57:12 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:12 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:12.275949 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:12.271287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:49049)
0806 12:57:12.271424 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:12.271438 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:12.271497 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:12.271606 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:12.271607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:12.271808 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:12.271878 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:12.271879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.272593 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.272594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:12.273150 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:12.273152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.273466 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.273467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:12.273514 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:12.273955 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:12.273959 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:12.275860 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
W20260806 12:57:12.276103 1868 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:12.271339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:49049 (local address 127.24.83.62:45843)
0806 12:57:12.271415 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:12.271417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:12.271500 (+ 83us) server_negotiation.cc:407] Connection header received
0806 12:57:12.271537 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:12.271538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:12.271548 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:12.271600 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:12.272014 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.272495 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.273226 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.273402 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.273450 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:12.276029 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:49049: BlockingRecv error: recv got EOF from 127.24.83.2:49049 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 12:57:12 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:12 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:12.410841 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:12.405980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:56973)
0806 12:57:12.406132 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:12.406143 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:12.406233 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:12.406364 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:12.406365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:12.406533 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:12.406602 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:12.406603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.407239 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.407239 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:12.407859 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:12.407861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.408242 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.408243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:12.408284 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:12.408773 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:12.408778 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:12.410728 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
W20260806 12:57:12.410872 1868 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:12.406032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:56973 (local address 127.24.83.62:45843)
0806 12:57:12.406093 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:12.406095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:12.406229 (+ 134us) server_negotiation.cc:407] Connection header received
0806 12:57:12.406293 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:12.406294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:12.406305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:12.406359 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:12.406659 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.407173 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.408001 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:12.408180 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:12.408230 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:12.410807 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:56973: BlockingRecv error: recv got EOF from 127.24.83.3:56973 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 12:57:13 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:13 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:13.154665 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:13.150017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:42597)
0806 12:57:13.150177 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:13.150193 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:13.150247 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:13.150445 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:13.150446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:13.150622 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:13.150675 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:13.150676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.151336 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.151337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:13.151882 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:13.151884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.152266 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.152267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:13.152323 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:13.152774 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:13.152778 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:13.154598 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
W20260806 12:57:13.154815 1872 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:13.150143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:42597 (local address 127.24.83.62:45843)
0806 12:57:13.150302 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:13.150306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:13.150315 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:57:13.150339 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:13.150339 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:13.150351 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:13.150431 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:13.150744 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.151238 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.152016 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.152201 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.152251 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:13.154713 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:42597: BlockingRecv error: recv got EOF from 127.24.83.1:42597 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
Aug 06 12:57:13 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:13 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:13.280676 1874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:13.276389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:57641)
0806 12:57:13.276541 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:13.276558 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:13.276613 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:13.276719 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:13.276720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:13.276887 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:13.276937 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:13.276939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.277548 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.277548 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:13.278065 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:13.278066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.278365 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.278366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:13.278404 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:13.278836 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:13.278840 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:13.280617 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
W20260806 12:57:13.280809 1872 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:13.276440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:57641 (local address 127.24.83.62:45843)
0806 12:57:13.276525 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:13.276528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:13.276616 (+ 88us) server_negotiation.cc:407] Connection header received
0806 12:57:13.276656 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:13.276657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:13.276667 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:13.276714 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:13.276995 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.277483 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.278133 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.278307 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.278355 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:13.280727 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:57641: BlockingRecv error: recv got EOF from 127.24.83.2:57641 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 12:57:13 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:13 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:13.415978 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:13.411323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:49023)
0806 12:57:13.411492 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:13.411512 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:13.411570 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:13.411696 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:13.411697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:13.411857 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:13.411952 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:13.411954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.412564 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.412564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:13.413078 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:13.413079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.413407 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.413408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:13.413447 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:13.413949 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:13.413954 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:13.415857 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
W20260806 12:57:13.416028 1872 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:13.411464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:49023 (local address 127.24.83.62:45843)
0806 12:57:13.411526 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:13.411528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:13.411555 (+ 27us) server_negotiation.cc:407] Connection header received
0806 12:57:13.411613 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:13.411614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:13.411627 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:13.411677 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:13.412009 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.412499 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.413141 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:13.413333 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:13.413386 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:13.415961 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:49023: BlockingRecv error: recv got EOF from 127.24.83.3:49023 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 12:57:14 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:14 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:14.160218 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:14.155169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:46379)
0806 12:57:14.155317 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:14.155329 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:14.155389 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:14.155537 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:14.155537 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:14.155715 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:14.155769 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:14.155770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.156455 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.156456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:14.157392 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:14.157396 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.157782 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.157784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:14.157846 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:14.158297 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:14.158301 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:14.160157 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
W20260806 12:57:14.160322 1877 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:14.155261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:46379 (local address 127.24.83.62:45843)
0806 12:57:14.155382 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:14.155384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:14.155394 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:57:14.155452 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:14.155453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:14.155464 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:14.155515 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:14.155843 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.156353 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.157473 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.157710 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.157771 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:14.160246 (+ 2475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:46379: BlockingRecv error: recv got EOF from 127.24.83.1:46379 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
Aug 06 12:57:14 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:14 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:14.285475 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:14.281143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:59035)
0806 12:57:14.281294 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:14.281311 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:14.281366 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:14.281491 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:14.281492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:14.281647 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:14.281697 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:14.281698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.282320 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.282320 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:14.282830 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:14.282831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.283140 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.283141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:14.283181 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:14.283627 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:14.283631 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:14.285378 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
W20260806 12:57:14.285531 1877 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:14.281200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:59035 (local address 127.24.83.62:45843)
0806 12:57:14.281262 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:14.281264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:14.281370 (+ 106us) server_negotiation.cc:407] Connection header received
0806 12:57:14.281409 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:14.281410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:14.281421 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:14.281472 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:14.281773 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.282254 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.282899 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.283080 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.283129 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:14.285468 (+ 2339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:59035: BlockingRecv error: recv got EOF from 127.24.83.2:59035 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 12:57:14 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:14 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:14.420818 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:14.416488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:35511)
0806 12:57:14.416645 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:14.416655 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:14.416710 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:14.416833 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:14.416834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:14.417001 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:14.417050 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:14.417052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.417652 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.417653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:14.418161 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:14.418163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.418471 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.418472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:14.418512 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:14.418958 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:14.418962 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:14.420721 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":102,"threads_started":1}
W20260806 12:57:14.420872 1877 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:14.416540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:35511 (local address 127.24.83.62:45843)
0806 12:57:14.416601 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:14.416603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:14.416709 (+ 106us) server_negotiation.cc:407] Connection header received
0806 12:57:14.416751 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:14.416752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:14.416763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:14.416827 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:14.417105 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.417586 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.418229 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:14.418410 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:14.418460 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:14.420807 (+ 2347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:35511: BlockingRecv error: recv got EOF from 127.24.83.3:35511 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 12:57:15 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:15 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:15.164915 1880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:15.160639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:60963)
0806 12:57:15.160771 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:15.160780 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:15.160832 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:15.160964 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:15.160965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:15.161133 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:15.161183 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:15.161185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.161821 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.161822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:15.162366 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:15.162367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.162711 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.162711 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:15.162752 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:15.163172 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:15.163176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:15.164854 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
W20260806 12:57:15.165020 1881 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:15.160708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:60963 (local address 127.24.83.62:45843)
0806 12:57:15.160817 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:15.160819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:15.160827 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:15.160884 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:15.160885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:15.160895 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:15.160942 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:15.161241 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.161744 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.162451 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.162646 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.162698 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:15.164948 (+ 2250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:60963: BlockingRecv error: recv got EOF from 127.24.83.1:60963 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":47,"threads_started":1}
Aug 06 12:57:15 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:15 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:15.290791 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:15.285976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:54493)
0806 12:57:15.286166 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:15.286177 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:15.286247 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:15.286421 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:15.286422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:15.286610 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:15.286657 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:15.286658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.287498 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.287500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:15.288138 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:15.288141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.288496 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.288497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:15.288556 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:15.288989 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:15.288994 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:15.290698 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
W20260806 12:57:15.290846 1881 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:15.286076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:54493 (local address 127.24.83.62:45843)
0806 12:57:15.286149 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:15.286152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:15.286226 (+ 74us) server_negotiation.cc:407] Connection header received
0806 12:57:15.286312 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:15.286313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:15.286327 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:15.286413 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:15.286735 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.287408 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.288219 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.288419 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.288498 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:15.290784 (+ 2286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:54493: BlockingRecv error: recv got EOF from 127.24.83.2:54493 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 12:57:15 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:15 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:15.425801 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:15.421313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:44305)
0806 12:57:15.421483 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:15.421493 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:15.421581 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:15.421733 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:15.421733 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:15.421915 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:15.421959 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:15.421961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.422618 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.422619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:15.423155 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:15.423156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.423509 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.423510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:15.423575 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:15.424041 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:15.424046 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:15.425709 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
W20260806 12:57:15.425868 1881 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:15.421382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:44305 (local address 127.24.83.62:45843)
0806 12:57:15.421463 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:15.421466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:15.421561 (+ 95us) server_negotiation.cc:407] Connection header received
0806 12:57:15.421642 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:15.421643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:15.421663 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:15.421728 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:15.422037 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.422535 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.423247 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:15.423439 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:15.423510 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:15.425802 (+ 2292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:44305: BlockingRecv error: recv got EOF from 127.24.83.3:44305 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 12:57:16 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:16 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:16.175160 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:16.165449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:53503)
0806 12:57:16.165619 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:16.165633 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:16.165696 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:16.165845 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:16.165846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:16.166008 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:16.166059 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:16.166061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.166702 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.166702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:16.167235 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:16.167237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.167563 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.167564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:16.167603 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:16.173388 (+ 5785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:16.173394 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:16.175092 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":72,"threads_started":1}
W20260806 12:57:16.175284 1885 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:16.165503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:53503 (local address 127.24.83.62:45843)
0806 12:57:16.165688 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:16.165690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:16.165698 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:16.165756 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:16.165756 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:16.165767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:16.165820 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:16.166125 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.166634 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.167307 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.167501 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.167552 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:16.175197 (+ 7645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:53503: BlockingRecv error: recv got EOF from 127.24.83.1:53503 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":98,"threads_started":1}
Aug 06 12:57:16 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:16 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:16.296100 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:16.291434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:57629)
0806 12:57:16.291600 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:16.291618 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:16.291697 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:16.291843 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:16.291844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:16.292104 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:16.292156 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:16.292158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.292790 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.292790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:16.293329 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:16.293330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.293644 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.293645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:16.293694 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:16.294144 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:16.294148 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:16.296028 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
W20260806 12:57:16.296190 1885 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:16.291513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:57629 (local address 127.24.83.62:45843)
0806 12:57:16.291563 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:16.291566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:16.291679 (+ 113us) server_negotiation.cc:407] Connection header received
0806 12:57:16.291759 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:16.291759 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:16.291772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:16.291833 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:16.292223 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.292720 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.293399 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.293582 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.293632 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:16.296123 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:57629: BlockingRecv error: recv got EOF from 127.24.83.2:57629 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 12:57:16 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:16 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:16.430814 1887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:16.426276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:54771)
0806 12:57:16.426442 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:16.426460 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:16.426520 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:16.426643 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:16.426643 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:16.426810 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:16.426859 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:16.426861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.427498 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.427499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:16.428066 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:16.428068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.428381 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.428382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:16.428421 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:16.428941 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:16.428946 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:16.430699 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
W20260806 12:57:16.430871 1885 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:16.426331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:54771 (local address 127.24.83.62:45843)
0806 12:57:16.426395 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:16.426398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:16.426518 (+ 120us) server_negotiation.cc:407] Connection header received
0806 12:57:16.426563 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:16.426564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:16.426577 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:16.426628 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:16.426922 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.427412 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.428137 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:16.428318 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:16.428370 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:16.430804 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:54771: BlockingRecv error: recv got EOF from 127.24.83.3:54771 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 12:57:17 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:17 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:17.180143 1888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:17.175680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:50801)
0806 12:57:17.175823 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:17.175834 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:17.175935 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:17.176090 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:17.176091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:17.176249 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:17.176299 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:17.176301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.176927 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.176927 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:17.177432 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:17.177433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.177742 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.177743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:17.177781 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:17.178240 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:17.178245 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:17.180047 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
W20260806 12:57:17.180210 1889 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:17.175729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:50801 (local address 127.24.83.62:45843)
0806 12:57:17.175876 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:17.175878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:17.175914 (+ 36us) server_negotiation.cc:407] Connection header received
0806 12:57:17.176001 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:17.176001 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:17.176012 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:17.176073 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:17.176362 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.176860 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.177499 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.177682 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.177731 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:17.180136 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:50801: BlockingRecv error: recv got EOF from 127.24.83.1:50801 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
Aug 06 12:57:17 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:17 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:17.301241 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:17.296583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:58973)
0806 12:57:17.296758 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:17.296771 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:17.296863 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:17.297013 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:17.297014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:17.297191 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:17.297240 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:17.297242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.297947 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.297949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:17.298489 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:17.298492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.298893 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.298894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:17.298959 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:17.299398 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:17.299402 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:17.301141 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":84,"threads_started":1}
W20260806 12:57:17.301293 1889 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:17.296674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:58973 (local address 127.24.83.62:45843)
0806 12:57:17.296752 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:17.296754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:17.296841 (+ 87us) server_negotiation.cc:407] Connection header received
0806 12:57:17.296940 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:17.296941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:17.296954 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:17.297009 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:17.297347 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.297853 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.298580 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.298793 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.298890 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:17.301225 (+ 2335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:58973: BlockingRecv error: recv got EOF from 127.24.83.2:58973 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 12:57:17 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:17 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:17.436131 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:17.431368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:46875)
0806 12:57:17.431511 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:17.431541 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:17.431600 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:17.431770 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:17.431772 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:17.431968 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:17.432036 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:17.432038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.432766 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.432767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:17.433346 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:17.433348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.433755 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.433757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:17.433814 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:17.434264 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:17.434268 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:17.436070 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":105,"threads_started":1}
W20260806 12:57:17.436290 1889 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:17.431433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:46875 (local address 127.24.83.62:45843)
0806 12:57:17.431519 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:17.431522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:17.431601 (+ 79us) server_negotiation.cc:407] Connection header received
0806 12:57:17.431661 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:17.431661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:17.431675 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:17.431716 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:17.432120 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.432669 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.433431 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:17.433625 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:17.433708 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:17.436186 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:46875: BlockingRecv error: recv got EOF from 127.24.83.3:46875 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 12:57:18 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:18 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:18.184921 1892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:18.180645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:59799)
0806 12:57:18.180802 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:18.180811 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:18.180855 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:18.181034 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:18.181034 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:18.181194 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:18.181243 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:18.181245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.181896 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.181897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:18.182402 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:18.182403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.182717 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.182717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:18.182755 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:18.183182 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:18.183186 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:18.184856 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":72,"threads_started":1}
W20260806 12:57:18.185034 1893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:18.180700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:59799 (local address 127.24.83.62:45843)
0806 12:57:18.180827 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:18.180828 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:18.180840 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:57:18.180915 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:18.180915 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:18.180944 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:18.181011 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:18.181304 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.181827 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.182469 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.182657 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.182705 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:18.184947 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:59799: BlockingRecv error: recv got EOF from 127.24.83.1:59799 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
Aug 06 12:57:18 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:18 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:18.306641 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:18.301781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:41587)
0806 12:57:18.301988 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:18.302000 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:18.302074 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:18.302232 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:18.302233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:18.302427 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:18.302477 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:18.302479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.303178 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.303179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:18.303709 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:18.303711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.304137 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.304138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:18.304202 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:18.304715 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:18.304721 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:18.306549 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":103,"threads_started":1}
W20260806 12:57:18.306701 1893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:18.301878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:41587 (local address 127.24.83.62:45843)
0806 12:57:18.301977 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:18.301979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:18.302055 (+ 76us) server_negotiation.cc:407] Connection header received
0806 12:57:18.302136 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:18.302138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:18.302149 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:18.302223 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:18.302581 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.303082 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.303797 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.304060 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.304138 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:18.306639 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:41587: BlockingRecv error: recv got EOF from 127.24.83.2:41587 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 12:57:18 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:18 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:18.440881 1895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:18.436573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:39783)
0806 12:57:18.436691 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:18.436708 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:18.436767 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:18.436897 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:18.436898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:18.437056 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:18.437104 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:18.437106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.437710 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.437711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:18.438215 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:18.438217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.438527 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.438528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:18.438568 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:18.438982 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:18.438987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:18.440784 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
W20260806 12:57:18.440932 1893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:18.436639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:39783 (local address 127.24.83.62:45843)
0806 12:57:18.436714 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:18.436716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:18.436750 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:57:18.436818 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:18.436819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:18.436831 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:18.436880 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:18.437160 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.437645 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.438286 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:18.438466 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:18.438515 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:18.440861 (+ 2346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:39783: BlockingRecv error: recv got EOF from 127.24.83.3:39783 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 12:57:19 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:19 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:19.190214 1896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:19.185538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:45525)
0806 12:57:19.185728 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:19.185740 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:19.185794 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:19.185950 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:19.185950 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:19.186119 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:19.186170 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:19.186172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.186787 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.186788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:19.187320 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:19.187321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.187722 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.187723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:19.187786 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:19.188350 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:19.188355 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:19.190108 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":101,"threads_started":1}
W20260806 12:57:19.190272 1897 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:19.185681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:45525 (local address 127.24.83.62:45843)
0806 12:57:19.185830 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:19.185832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:19.185840 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:19.185862 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:19.185863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:19.185872 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:19.185931 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:19.186235 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.186720 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.187380 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.187613 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.187689 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:19.190199 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:45525: BlockingRecv error: recv got EOF from 127.24.83.1:45525 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
Aug 06 12:57:19 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:19 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:19.311996 1898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:19.307200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:33379)
0806 12:57:19.307372 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:19.307384 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:19.307453 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:19.307577 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:19.307577 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:19.307740 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:19.307792 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:19.307795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.308589 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.308590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:19.309114 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:19.309115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.309543 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.309544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:19.309586 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:19.310083 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:19.310088 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:19.311883 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":124,"threads_started":1}
W20260806 12:57:19.312088 1897 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:19.307268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:33379 (local address 127.24.83.62:45843)
0806 12:57:19.307317 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:19.307319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:19.307453 (+ 134us) server_negotiation.cc:407] Connection header received
0806 12:57:19.307497 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:19.307497 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:19.307511 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:19.307560 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:19.307858 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.308523 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.309184 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.309452 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.309536 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:19.312023 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:33379: BlockingRecv error: recv got EOF from 127.24.83.2:33379 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 12:57:19 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:19 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:19.445838 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:19.441338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:48643)
0806 12:57:19.441472 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:19.441483 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:19.441548 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:19.441668 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:19.441669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:19.441821 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:19.441900 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:19.441902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.442514 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.442514 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:19.443041 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:19.443043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.443346 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.443347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:19.443400 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:19.443841 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:19.443845 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:19.445741 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
W20260806 12:57:19.445919 1897 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:19.441418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:48643 (local address 127.24.83.62:45843)
0806 12:57:19.441481 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:19.441483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:19.441553 (+ 70us) server_negotiation.cc:407] Connection header received
0806 12:57:19.441596 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:19.441597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:19.441609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:19.441649 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:19.441962 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.442449 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.443109 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:19.443285 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:19.443334 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:19.445841 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:48643: BlockingRecv error: recv got EOF from 127.24.83.3:48643 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 12:57:20 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:20 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:20.195353 1900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:20.190765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:54969)
0806 12:57:20.190912 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:20.190928 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:20.190983 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:20.191119 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:20.191120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:20.191282 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:20.191333 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:20.191334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.192014 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.192015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:20.192549 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:20.192551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.192875 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.192875 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:20.192917 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:20.193352 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:20.193356 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:20.195283 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
W20260806 12:57:20.195462 1901 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:20.190851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:54969 (local address 127.24.83.62:45843)
0806 12:57:20.190982 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:20.190987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:20.191016 (+ 29us) server_negotiation.cc:407] Connection header received
0806 12:57:20.191036 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:20.191037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:20.191047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:20.191097 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:20.191401 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.191949 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.192621 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.192812 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.192863 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:20.195383 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:54969: BlockingRecv error: recv got EOF from 127.24.83.1:54969 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
Aug 06 12:57:20 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:20 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:20.316910 1902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:20.312474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:51759)
0806 12:57:20.312617 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:20.312627 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:20.312687 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:20.312817 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:20.312818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:20.312989 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:20.313041 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:20.313043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.313669 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.313670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:20.314221 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:20.314223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.314563 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.314564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:20.314606 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:20.315046 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:20.315050 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:20.316840 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
W20260806 12:57:20.317130 1901 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:20.312561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:51759 (local address 127.24.83.62:45843)
0806 12:57:20.312627 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:20.312629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:20.312685 (+ 56us) server_negotiation.cc:407] Connection header received
0806 12:57:20.312731 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:20.312732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:20.312743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:20.312808 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:20.313104 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.313602 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.314302 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.314495 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.314551 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:20.316990 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:51759: BlockingRecv error: recv got EOF from 127.24.83.2:51759 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 12:57:20 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:20 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:20.451134 1903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:20.446276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:37461)
0806 12:57:20.446444 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:20.446455 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:20.446517 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:20.446638 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:20.446639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:20.446796 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:20.446845 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:20.446847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.447664 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.447664 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:20.448268 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:20.448269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.448577 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.448578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:20.448618 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:20.449038 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:20.449042 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:20.451076 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
W20260806 12:57:20.451326 1901 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:20.446335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:37461 (local address 127.24.83.62:45843)
0806 12:57:20.446401 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:20.446403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:20.446515 (+ 112us) server_negotiation.cc:407] Connection header received
0806 12:57:20.446560 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:20.446560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:20.446572 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:20.446621 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:20.446903 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.447575 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.448335 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:20.448513 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:20.448567 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:20.451245 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:37461: BlockingRecv error: recv got EOF from 127.24.83.3:37461 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 12:57:21 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:21 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:21.205492 1904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:21.195997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:52659)
0806 12:57:21.196161 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:21.196171 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:21.196229 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:21.196367 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:21.196368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:21.196540 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:21.196589 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:21.196591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.197210 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.197211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:21.197815 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:21.197818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.198198 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.198199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:21.198261 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:21.203482 (+ 5221us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:21.203487 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:21.205377 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
W20260806 12:57:21.205549 1905 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:21.196105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:52659 (local address 127.24.83.62:45843)
0806 12:57:21.196210 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:21.196212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:21.196229 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:57:21.196276 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:21.196276 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:21.196292 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:21.196349 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:21.196655 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.197147 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.197909 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.198111 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.198197 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:21.205471 (+ 7274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:52659: BlockingRecv error: recv got EOF from 127.24.83.1:52659 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":53,"threads_started":1}
Aug 06 12:57:21 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:21 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:21.322012 1906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:21.317430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:45667)
0806 12:57:21.317600 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:21.317612 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:21.317697 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:21.317857 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:21.317859 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:21.318049 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:21.318097 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:21.318099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.318785 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.318786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:21.319315 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:21.319316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.319696 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.319696 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:21.319762 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:21.320231 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:21.320235 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:21.321915 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
W20260806 12:57:21.322067 1905 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:21.317509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:45667 (local address 127.24.83.62:45843)
0806 12:57:21.317585 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:21.317588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:21.317681 (+ 93us) server_negotiation.cc:407] Connection header received
0806 12:57:21.317760 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:21.317761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:21.317775 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:21.317849 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:21.318179 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.318688 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.319406 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.319603 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.319694 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:21.322001 (+ 2307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:45667: BlockingRecv error: recv got EOF from 127.24.83.2:45667 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 12:57:21 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:21 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:21.456328 1907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:21.451657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:39781)
0806 12:57:21.451813 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:21.451828 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:21.451910 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:21.452033 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:21.452033 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:21.452198 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:21.452247 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:21.452248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.452853 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.452854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:21.453373 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:21.453375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.453676 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.453677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:21.453724 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:21.454239 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:21.454244 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:21.456216 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
W20260806 12:57:21.456382 1905 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:21.451803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:39781 (local address 127.24.83.62:45843)
0806 12:57:21.451866 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:21.451868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:21.451878 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:57:21.451964 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:21.451965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:21.451977 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:21.452021 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:21.452297 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.452786 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.453439 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:21.453615 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:21.453664 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:21.456313 (+ 2649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:39781: BlockingRecv error: recv got EOF from 127.24.83.3:39781 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 12:57:22 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:22 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:22.210867 1908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:22.205990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:41935)
0806 12:57:22.206155 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:22.206168 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:22.206235 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:22.206406 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:22.206406 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:22.206577 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:22.206626 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:22.206627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.207317 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.207318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:22.207879 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:22.207881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.208334 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.208335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:22.208379 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:22.208882 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:22.208886 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:22.210810 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
W20260806 12:57:22.211009 1909 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:22.206101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:41935 (local address 127.24.83.62:45843)
0806 12:57:22.206254 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:22.206256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:22.206264 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:22.206301 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:22.206302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:22.206316 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:22.206395 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:22.206690 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.207256 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.208025 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.208271 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.208323 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:22.210930 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:41935: BlockingRecv error: recv got EOF from 127.24.83.1:41935 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
Aug 06 12:57:22 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:22 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:22.327082 1910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:22.322474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:34583)
0806 12:57:22.322646 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:22.322657 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:22.322724 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:22.322894 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:22.322895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:22.323087 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:22.323133 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:22.323135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.323831 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.323832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:22.324397 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:22.324398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.324786 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.324786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:22.324847 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:22.325303 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:22.325307 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:22.326989 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
W20260806 12:57:22.327147 1909 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:22.322546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:34583 (local address 127.24.83.62:45843)
0806 12:57:22.322625 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:22.322628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:22.322705 (+ 77us) server_negotiation.cc:407] Connection header received
0806 12:57:22.322804 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:22.322805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:22.322822 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:22.322885 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:22.323219 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.323725 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.324487 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.324696 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.324783 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:22.327083 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:34583: BlockingRecv error: recv got EOF from 127.24.83.2:34583 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 12:57:22 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:22 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:22.461714 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:22.456800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:57589)
0806 12:57:22.456987 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:22.456999 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:22.457067 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:22.457232 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:22.457234 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:22.457423 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:22.457468 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:22.457469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.458202 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.458203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:22.458931 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:22.458935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.459284 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.459285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:22.459330 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:22.459817 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:22.459822 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:22.461616 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":112,"threads_started":1}
W20260806 12:57:22.461750 1909 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:22.456875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:57589 (local address 127.24.83.62:45843)
0806 12:57:22.456977 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:22.456980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:22.457045 (+ 65us) server_negotiation.cc:407] Connection header received
0806 12:57:22.457129 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:22.457130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:22.457144 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:22.457208 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:22.457546 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.458071 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.459020 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:22.459215 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:22.459270 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:22.461689 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:57589: BlockingRecv error: recv got EOF from 127.24.83.3:57589 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 12:57:23 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:23 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:23.215629 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:23.211273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:55791)
0806 12:57:23.211439 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:23.211450 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:23.211510 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:23.211659 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:23.211660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:23.211831 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:23.211878 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:23.211879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.212540 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.212541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:23.213054 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:23.213056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.213383 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.213389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:23.213428 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:23.213888 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:23.213893 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:23.215557 (+ 1664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
W20260806 12:57:23.215732 1913 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:23.211351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:55791 (local address 127.24.83.62:45843)
0806 12:57:23.211465 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:23.211467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:23.211494 (+ 27us) server_negotiation.cc:407] Connection header received
0806 12:57:23.211569 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:23.211570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:23.211581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:23.211638 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:23.211977 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.212474 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.213126 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.213318 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.213368 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:23.215654 (+ 2286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:55791: BlockingRecv error: recv got EOF from 127.24.83.1:55791 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
Aug 06 12:57:23 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:23 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:23.332535 1914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:23.327613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:44895)
0806 12:57:23.327824 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:23.327836 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:23.327956 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:23.328176 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:23.328177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:23.328365 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:23.328413 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:23.328415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.329094 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.329095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:23.329629 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:23.329631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.330015 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.330017 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:23.330083 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:23.330604 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:23.330608 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:23.332474 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":111,"threads_started":1}
W20260806 12:57:23.332671 1913 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:23.327730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:44895 (local address 127.24.83.62:45843)
0806 12:57:23.327807 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:23.327809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:23.327910 (+ 101us) server_negotiation.cc:407] Connection header received
0806 12:57:23.328055 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:23.328056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:23.328089 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:23.328142 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:23.328502 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.329005 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.329718 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.329933 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.330015 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:23.332590 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:44895: BlockingRecv error: recv got EOF from 127.24.83.2:44895 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 12:57:23 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:23 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:23.468107 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:23.462236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:33153)
0806 12:57:23.462446 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:23.462461 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:23.462548 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:23.462748 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:23.462749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:23.463030 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:23.463110 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:23.463112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.464115 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.464120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:23.464668 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:23.464669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.465091 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.465092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:23.465185 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:23.465722 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:23.465727 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:23.468036 (+ 2309us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":61,"threads_started":1}
W20260806 12:57:23.468205 1913 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:23.462314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:33153 (local address 127.24.83.62:45843)
0806 12:57:23.462412 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:23.462415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:23.462524 (+ 109us) server_negotiation.cc:407] Connection header received
0806 12:57:23.462613 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:23.462615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:23.462630 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:23.462725 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:23.463192 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.463997 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.464813 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:23.465023 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:23.465087 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:23.468115 (+ 3028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:33153: BlockingRecv error: recv got EOF from 127.24.83.3:33153 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 12:57:24 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:24 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:24.220520 1917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:24.216122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:36831)
0806 12:57:24.216262 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:24.216281 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:24.216330 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:24.216473 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:24.216474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:24.216646 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:24.216696 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:24.216697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.217336 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.217337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:24.217857 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:24.217858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.218183 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.218184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:24.218223 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:24.218663 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:24.218667 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:24.220446 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
W20260806 12:57:24.220624 1918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:24.216204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:36831 (local address 127.24.83.62:45843)
0806 12:57:24.216341 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:24.216343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:24.216351 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:24.216373 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:24.216374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:24.216384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:24.216455 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:24.216764 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.217259 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.217927 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.218119 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.218168 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:24.220548 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:36831: BlockingRecv error: recv got EOF from 127.24.83.1:36831 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
Aug 06 12:57:24 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:24 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:24.337236 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:24.332962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:49865)
0806 12:57:24.333116 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:24.333127 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:24.333175 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:24.333348 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:24.333349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:24.333529 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:24.333578 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:24.333579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.334215 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.334216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:24.334730 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:24.334732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.335047 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.335047 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:24.335087 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:24.335515 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:24.335519 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:24.337178 (+ 1659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
W20260806 12:57:24.337357 1918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:24.333087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:49865 (local address 127.24.83.62:45843)
0806 12:57:24.333152 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:24.333154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:24.333165 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:57:24.333243 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:24.333244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:24.333254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:24.333307 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:24.333655 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.334149 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.334804 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.334985 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.335036 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:24.337271 (+ 2235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:49865: BlockingRecv error: recv got EOF from 127.24.83.2:49865 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 12:57:24 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:24 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:24.473316 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:24.468677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:44941)
0806 12:57:24.468846 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:24.468857 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:24.468930 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:24.469071 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:24.469071 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:24.469260 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:24.469317 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:24.469319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.470026 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.470028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:24.470557 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:24.470559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.470929 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.470930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:24.470993 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:24.471446 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:24.471450 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:24.473253 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
W20260806 12:57:24.473452 1918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:24.468744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:44941 (local address 127.24.83.62:45843)
0806 12:57:24.468821 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:24.468823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:24.468922 (+ 99us) server_negotiation.cc:407] Connection header received
0806 12:57:24.468986 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:24.468987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:24.469000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:24.469041 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:24.469381 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.469939 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.470646 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:24.470842 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:24.470926 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:24.473346 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:44941: BlockingRecv error: recv got EOF from 127.24.83.3:44941 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 12:57:25 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:25 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:25.225733 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:25.221114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:60635)
0806 12:57:25.221262 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:25.221272 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:25.221331 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:25.221481 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:25.221482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:25.221646 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:25.221696 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:25.221697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.222318 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.222319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:25.222852 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:25.222853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.223165 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.223166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:25.223203 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:25.223650 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:25.223654 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:25.225667 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
W20260806 12:57:25.225837 1922 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:25.221197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:60635 (local address 127.24.83.62:45843)
0806 12:57:25.221333 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:25.221335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:25.221368 (+ 33us) server_negotiation.cc:407] Connection header received
0806 12:57:25.221397 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:25.221397 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:25.221408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:25.221458 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:25.221763 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.222249 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.222920 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.223105 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.223154 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:25.225762 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:60635: BlockingRecv error: recv got EOF from 127.24.83.1:60635 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
Aug 06 12:57:25 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:25 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:25.342036 1923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:25.337743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:49081)
0806 12:57:25.337897 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:25.337908 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:25.337967 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:25.338091 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:25.338092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:25.338256 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:25.338314 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:25.338316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.338934 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.338935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:25.339454 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:25.339456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.339768 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.339769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:25.339809 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:25.340322 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:25.340326 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:25.341970 (+ 1644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
W20260806 12:57:25.342133 1922 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:25.337798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:49081 (local address 127.24.83.62:45843)
0806 12:57:25.337869 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:25.337871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:25.337969 (+ 98us) server_negotiation.cc:407] Connection header received
0806 12:57:25.338012 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:25.338013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:25.338023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:25.338073 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:25.338378 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.338871 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.339525 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.339707 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.339756 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:25.342065 (+ 2309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:49081: BlockingRecv error: recv got EOF from 127.24.83.2:49081 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 12:57:25 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:25 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.3@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:25.478118 1924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:25.473766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.3:53905)
0806 12:57:25.473930 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:25.473940 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:25.473995 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:25.474098 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:25.474099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:25.474262 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:25.474310 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:25.474311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.474921 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.474922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:25.475444 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:25.475446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.475752 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.475752 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:25.475794 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:25.476264 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:25.476268 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:25.478020 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":109,"threads_started":1}
W20260806 12:57:25.478161 1922 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:25.473838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:53905 (local address 127.24.83.62:45843)
0806 12:57:25.473896 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:25.473898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:25.473992 (+ 94us) server_negotiation.cc:407] Connection header received
0806 12:57:25.474036 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:25.474036 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:25.474049 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:25.474094 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:25.474362 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.474846 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.475512 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:25.475688 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:25.475736 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:25.478099 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.3:53905: BlockingRecv error: recv got EOF from 127.24.83.3:53905 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 12:57:26 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:26 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.1@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:26.235136 1925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:26.226397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.1:57473)
0806 12:57:26.226542 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:26.226571 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:26.226626 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:26.226759 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:26.226760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:26.226914 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:26.226964 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:26.226966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.227601 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.227602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:26.228417 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:26.228420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.228748 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.228749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:26.228788 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:26.233328 (+ 4540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:26.233335 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:26.235075 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
W20260806 12:57:26.235288 1926 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:26.226485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:57473 (local address 127.24.83.62:45843)
0806 12:57:26.226627 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:26.226628 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:26.226637 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:57:26.226662 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:26.226663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:26.226673 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:26.226737 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:26.227028 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.227526 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.228483 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.228681 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.228735 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:26.235193 (+ 6458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.1:57473: BlockingRecv error: recv got EOF from 127.24.83.1:57473 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
Aug 06 12:57:26 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:57:26 dist-test-slave-zkpd krb5kdc[847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.83.2@KRBTEST.COM for oryx/127.24.83.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:57:26.347415 1927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:26.342604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45843 (local address 127.24.83.2:55093)
0806 12:57:26.342747 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:26.342759 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:26.342848 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:26.343006 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:26.343007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:26.343193 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:26.343242 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:26.343243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.343956 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.343957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:26.344493 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:26.344495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.344881 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.344882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:26.344945 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:26.345402 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:26.345406 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:26.347329 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.83.62:45843: Server oryx/127.24.83.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
W20260806 12:57:26.347496 1926 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:26.342796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:55093 (local address 127.24.83.62:45843)
0806 12:57:26.342869 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:26.342871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:26.342880 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:57:26.342909 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:26.342910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:26.342922 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:26.342997 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:26.343333 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.343842 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.344582 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.344787 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.344877 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:26.347431 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.83.2:55093: BlockingRecv error: recv got EOF from 127.24.83.2:55093 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 12:57:26.400322 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 1444
I20260806 12:57:26.406049 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 1597
I20260806 12:57:26.411478 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 1731
I20260806 12:57:26.417554 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 1372
2026-08-06T12:57:26Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17458 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T12:57:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:57:26Z Disabled control of system clock
I20260806 12:57:26.445771 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33401
--webserver_interface=127.24.83.62
--webserver_port=0
--builtin_ntp_servers=127.24.83.20:36153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:33401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 12:57:26.527693 1931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:26.527878 1931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:26.527931 1931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:26.529026 1931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:57:26.529057 1931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:26.529069 1931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:26.529083 1931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:57:26.529094 1931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:57:26.530334 1931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:33401
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:26.530514 1931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:26.530730 1931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:57:26.536010 1931 server_base.cc:1034] running on GCE node
W20260806 12:57:26.535967 1936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:26.536027 1939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:26.535962 1937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:26.536396 1931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:26.536710 1931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:26.537868 1931 hybrid_clock.cc:648] HybridClock initialized: now 1786021046537864 us; error 40 us; skew 500 ppm
I20260806 12:57:26.539116 1931 webserver.cc:460] Webserver started at http://127.24.83.62:35863/ using document root <none> and password file <none>
I20260806 12:57:26.539410 1931 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:26.539484 1931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:26.539641 1931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:26.540753 1931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "6a0857bee4d94c67afa8d882931df3ff"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "d11185d649eaef13a4437fc4cfb28c0a"
server_key_iv: "a0a5ad8a271ca6f959a3b4d40c6d0871"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.541106 1931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "6a0857bee4d94c67afa8d882931df3ff"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "d11185d649eaef13a4437fc4cfb28c0a"
server_key_iv: "a0a5ad8a271ca6f959a3b4d40c6d0871"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.542327 1931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 12:57:26.543010 1945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:26.543188 1931 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:26.543291 1931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "6a0857bee4d94c67afa8d882931df3ff"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "d11185d649eaef13a4437fc4cfb28c0a"
server_key_iv: "a0a5ad8a271ca6f959a3b4d40c6d0871"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.543363 1931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:26.553790 1931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:26.554083 1931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:26.554230 1931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:26.558208 1931 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:33401
I20260806 12:57:26.558239 1997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:33401 every 8 connection(s)
I20260806 12:57:26.558593 1931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:57:26.559230 1998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:26.560029 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 1931
I20260806 12:57:26.560141 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:57:26.560348 24908 external_mini_cluster.cc:1428] Setting key fb3baffc63c0c5398e6955eee598a620
I20260806 12:57:26.562584 1998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff: Bootstrap starting.
I20260806 12:57:26.563308 1998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:26.563616 1998 log.cc:824] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:26.564394 1998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff: No bootstrap required, opened a new log
I20260806 12:57:26.565948 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:26.560872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40592 (local address 127.24.83.62:33401)
0806 12:57:26.561077 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:26.561085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:26.561108 (+ 23us) server_negotiation.cc:407] Connection header received
0806 12:57:26.561525 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:26.561531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:26.561589 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:26.561712 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:26.561934 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.562853 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.564026 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.564541 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.564634 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:26.565280 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:26.565284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:26.565284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:57:26.565453 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:26.565485 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:57:26.565496 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:26.565546 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:26.565655 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 12:57:26.565741 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260806 12:57:26.566735 1998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0857bee4d94c67afa8d882931df3ff" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33401 } }
I20260806 12:57:26.566869 1998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:26.566892 1998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a0857bee4d94c67afa8d882931df3ff, State: Initialized, Role: FOLLOWER
I20260806 12:57:26.567036 1998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0857bee4d94c67afa8d882931df3ff" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33401 } }
I20260806 12:57:26.567099 1998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:57:26.567140 1998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:57:26.567188 1998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:26.567726 1998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0857bee4d94c67afa8d882931df3ff" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33401 } }
I20260806 12:57:26.567850 1998 leader_election.cc:302] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a0857bee4d94c67afa8d882931df3ff; no voters:
I20260806 12:57:26.568037 1998 leader_election.cc:288] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:57:26.568076 2003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:26.568259 2003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [term 1 LEADER]: Becoming Leader. State: Replica: 6a0857bee4d94c67afa8d882931df3ff, State: Running, Role: LEADER
I20260806 12:57:26.568307 1998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:57:26.568392 2003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0857bee4d94c67afa8d882931df3ff" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33401 } }
I20260806 12:57:26.568748 2005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a0857bee4d94c67afa8d882931df3ff. Latest consensus state: current_term: 1 leader_uuid: "6a0857bee4d94c67afa8d882931df3ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0857bee4d94c67afa8d882931df3ff" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33401 } } }
I20260806 12:57:26.568864 2005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:26.569132 2012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:57:26.568742 2004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a0857bee4d94c67afa8d882931df3ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0857bee4d94c67afa8d882931df3ff" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33401 } } }
I20260806 12:57:26.569410 2004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:26.569640 2012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:57:26.571141 2012 catalog_manager.cc:1347] Generated new cluster ID: 10d5725a905148ab88f9a1413859e8e1
I20260806 12:57:26.571197 2012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:57:26.587496 2012 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:57:26.588194 2012 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:57:26.592221 2012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a0857bee4d94c67afa8d882931df3ff: Generated new TSK 0
I20260806 12:57:26.592392 2012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:57:26.596396 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33401
--builtin_ntp_servers=127.24.83.20:36153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 12:57:26.679183 2022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:26.679409 2022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:26.679443 2022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:26.680645 2022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:26.680714 2022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:26.680752 2022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:57:26.682106 2022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:26.682379 2022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:26.682673 2022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:26.688432 2027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:26.688441 2030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:26.688441 2028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:26.688865 2022 server_base.cc:1034] running on GCE node
I20260806 12:57:26.689082 2022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:26.689365 2022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:26.690539 2022 hybrid_clock.cc:648] HybridClock initialized: now 1786021046690519 us; error 35 us; skew 500 ppm
I20260806 12:57:26.691843 2022 webserver.cc:460] Webserver started at http://127.24.83.1:35879/ using document root <none> and password file <none>
I20260806 12:57:26.692122 2022 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:26.692193 2022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:26.692344 2022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:26.693368 2022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "449b7afabaf845d38bb968b4917fd652"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "64a54e2276f0142d5df4628e4babab5f"
server_key_iv: "b02af93c94eda2f9ffa6c542184d24e3"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.693745 2022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "449b7afabaf845d38bb968b4917fd652"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "64a54e2276f0142d5df4628e4babab5f"
server_key_iv: "b02af93c94eda2f9ffa6c542184d24e3"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.695029 2022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:57:26.695722 2036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:26.695947 2022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:57:26.696043 2022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "449b7afabaf845d38bb968b4917fd652"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "64a54e2276f0142d5df4628e4babab5f"
server_key_iv: "b02af93c94eda2f9ffa6c542184d24e3"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.696126 2022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:26.725788 2022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:26.726121 2022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:26.726280 2022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:26.726512 2022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:26.726854 2022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:26.726907 2022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:26.726948 2022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:26.726976 2022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:26.732905 2022 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:35007
I20260806 12:57:26.732949 2151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:35007 every 8 connection(s)
I20260806 12:57:26.733275 2022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:57:26.738435 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:26.733961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:52995 (local address 127.24.83.62:33401)
0806 12:57:26.734047 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:26.734049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:26.734291 (+ 242us) server_negotiation.cc:407] Connection header received
0806 12:57:26.734782 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:26.734783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:26.734798 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:26.734864 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:26.735520 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.736208 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.737174 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.737363 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.737422 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:26.738194 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:26.738195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:26.738195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:57:26.738199 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:26.738215 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:57:26.738218 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:26.738261 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:26.738350 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 12:57:26.738374 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 12:57:26.738610 2154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:26.733951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33401 (local address 127.24.83.1:52995)
0806 12:57:26.734158 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:26.734200 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:26.734701 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:26.734945 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:26.734948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:26.734957 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:26.735445 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:26.735450 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.736304 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.736305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:26.737084 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:26.737086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.737444 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.737444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:26.737549 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:26.738066 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:57:26.738067 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:26.738095 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:57:26.738117 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:26.738273 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:26.738277 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:26.738280 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:57:26.738330 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 12:57:26.738415 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260806 12:57:26.739122 2152 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33401
I20260806 12:57:26.739235 2152 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:26.739465 2152 heartbeater.cc:511] Master 127.24.83.62:33401 requested a full tablet report, sending...
I20260806 12:57:26.740010 1962 ts_manager.cc:194] Registered new tserver with Master: 449b7afabaf845d38bb968b4917fd652 (127.24.83.1:35007)
I20260806 12:57:26.740648 1962 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.1:52995
I20260806 12:57:26.741747 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 2022
I20260806 12:57:26.741861 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:57:26.742067 24908 external_mini_cluster.cc:1428] Setting key 4e8f64085cda3e0777de48a461818175
I20260806 12:57:26.742971 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33401
--builtin_ntp_servers=127.24.83.20:36153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 12:57:26.824414 2155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:26.824637 2155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:26.824671 2155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:26.825789 2155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:26.825887 2155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:26.825929 2155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:57:26.827299 2155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:26.827560 2155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:26.827880 2155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:26.832651 2160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:26.832631 2163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:26.832715 2161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:26.832823 2155 server_base.cc:1034] running on GCE node
I20260806 12:57:26.833026 2155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:26.833254 2155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:26.834398 2155 hybrid_clock.cc:648] HybridClock initialized: now 1786021046834386 us; error 28 us; skew 500 ppm
I20260806 12:57:26.835590 2155 webserver.cc:460] Webserver started at http://127.24.83.2:45479/ using document root <none> and password file <none>
I20260806 12:57:26.835824 2155 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:26.835868 2155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:26.836041 2155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:26.837062 2155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "a24b76265e814e10954b1aaf6ae59809"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "a4e0e757dca824421db4adad3f5ab872"
server_key_iv: "ba827c0af5b15bfc33cb54d3c573d16e"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.837440 2155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "a24b76265e814e10954b1aaf6ae59809"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "a4e0e757dca824421db4adad3f5ab872"
server_key_iv: "ba827c0af5b15bfc33cb54d3c573d16e"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.838779 2155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:57:26.839499 2169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:26.839673 2155 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:26.839767 2155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "a24b76265e814e10954b1aaf6ae59809"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "a4e0e757dca824421db4adad3f5ab872"
server_key_iv: "ba827c0af5b15bfc33cb54d3c573d16e"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.839854 2155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:26.853554 2155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:26.853832 2155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:26.853979 2155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:26.854187 2155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:26.854506 2155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:26.854559 2155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:26.854599 2155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:26.854626 2155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:26.860334 2155 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:40967
I20260806 12:57:26.860389 2284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:40967 every 8 connection(s)
I20260806 12:57:26.860709 2155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:57:26.865492 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:26.861435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:42123 (local address 127.24.83.62:33401)
0806 12:57:26.861487 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:26.861490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:26.861732 (+ 242us) server_negotiation.cc:407] Connection header received
0806 12:57:26.862168 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:26.862169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:26.862179 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:26.862225 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:26.862841 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.863317 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.864318 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.864506 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.864562 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:26.865263 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:26.865264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:26.865265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:57:26.865269 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:26.865283 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:57:26.865287 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:26.865329 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:26.865413 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 12:57:26.865434 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:57:26.865711 2287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:26.861423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33401 (local address 127.24.83.2:42123)
0806 12:57:26.861614 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:26.861646 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:26.862087 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:26.862281 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:26.862284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:26.862292 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:26.862771 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:26.862775 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.863403 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.863403 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:26.864233 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:26.864236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.864590 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.864590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:26.864691 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:26.865137 (+ 446us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:57:26.865137 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:26.865163 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:57:26.865183 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:26.865342 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:26.865346 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:26.865348 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 12:57:26.865403 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 12:57:26.865479 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260806 12:57:26.866284 2285 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33401
I20260806 12:57:26.866393 2285 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:26.866585 2285 heartbeater.cc:511] Master 127.24.83.62:33401 requested a full tablet report, sending...
I20260806 12:57:26.866968 1962 ts_manager.cc:194] Registered new tserver with Master: a24b76265e814e10954b1aaf6ae59809 (127.24.83.2:40967)
I20260806 12:57:26.867341 1962 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.2:42123
I20260806 12:57:26.867854 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 2155
I20260806 12:57:26.867982 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:57:26.868168 24908 external_mini_cluster.cc:1428] Setting key 8ecacd7df6820e68379e878715709258
I20260806 12:57:26.869112 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33401
--builtin_ntp_servers=127.24.83.20:36153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 12:57:26.950131 2288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:26.950320 2288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:26.950340 2288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:26.951388 2288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:26.951419 2288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:26.951442 2288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:57:26.952749 2288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:26.952940 2288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:26.953179 2288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:26.958192 2293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:26.958205 2296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:26.958205 2294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:26.958463 2288 server_base.cc:1034] running on GCE node
I20260806 12:57:26.958668 2288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:26.958915 2288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:26.960071 2288 hybrid_clock.cc:648] HybridClock initialized: now 1786021046960067 us; error 34 us; skew 500 ppm
I20260806 12:57:26.961266 2288 webserver.cc:460] Webserver started at http://127.24.83.3:36107/ using document root <none> and password file <none>
I20260806 12:57:26.961522 2288 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:26.961612 2288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:26.961750 2288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:26.962761 2288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "b3a97d54962d4984acc94e74d3370c16"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "9996bdb966c542c0dd37475e2d9a0d89"
server_key_iv: "e1a1b470fec01c8f595863666d6f5ce2"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.963147 2288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "b3a97d54962d4984acc94e74d3370c16"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "9996bdb966c542c0dd37475e2d9a0d89"
server_key_iv: "e1a1b470fec01c8f595863666d6f5ce2"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.964524 2288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:57:26.965210 2302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:26.965384 2288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:26.965487 2288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "b3a97d54962d4984acc94e74d3370c16"
format_stamp: "Formatted at 2026-08-06 12:57:26 on dist-test-slave-zkpd"
server_key: "9996bdb966c542c0dd37475e2d9a0d89"
server_key_iv: "e1a1b470fec01c8f595863666d6f5ce2"
server_key_version: "encryptionkey@0"
I20260806 12:57:26.965560 2288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:26.980429 2288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:26.980695 2288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:26.980849 2288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:26.981053 2288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:26.981362 2288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:26.981419 2288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:26.981458 2288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:26.981484 2288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:26.987214 2288 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:44731
I20260806 12:57:26.987262 2416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:44731 every 8 connection(s)
I20260806 12:57:26.987550 2288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:57:26.992202 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:26.988236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:43385 (local address 127.24.83.62:33401)
0806 12:57:26.988278 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:26.988280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:26.988475 (+ 195us) server_negotiation.cc:407] Connection header received
0806 12:57:26.988951 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:26.988952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:26.988961 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:26.989010 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:26.989592 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.990071 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.990968 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.991144 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.991192 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:26.991959 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:26.991960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:26.991960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:57:26.991965 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:26.991979 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:57:26.991983 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:26.992036 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:26.992126 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 12:57:26.992146 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 12:57:26.992404 2419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:26.988204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33401 (local address 127.24.83.3:43385)
0806 12:57:26.988378 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:26.988409 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:26.988864 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:26.989060 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:26.989062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:26.989070 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:26.989525 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:26.989528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.990146 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.990146 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:26.990896 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:26.990897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:26.991213 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:26.991213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:26.991315 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:26.991779 (+ 464us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:57:26.991780 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:26.991816 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:57:26.991836 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:26.992036 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:26.992039 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:26.992041 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 12:57:26.992120 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 12:57:26.992195 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":91,"threads_started":1}
I20260806 12:57:26.992897 2417 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33401
I20260806 12:57:26.992997 2417 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:26.993165 2417 heartbeater.cc:511] Master 127.24.83.62:33401 requested a full tablet report, sending...
I20260806 12:57:26.993557 1962 ts_manager.cc:194] Registered new tserver with Master: b3a97d54962d4984acc94e74d3370c16 (127.24.83.3:44731)
I20260806 12:57:26.993892 1962 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.3:43385
I20260806 12:57:26.994345 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 2288
I20260806 12:57:26.994441 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:57:26.994635 24908 external_mini_cluster.cc:1428] Setting key b3bc97934cef68eaf71d6d7407b027a3
I20260806 12:57:26.995357 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 12:57:26.997026 2001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:26.996606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40598 (local address 127.24.83.62:33401)
0806 12:57:26.996638 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:26.996640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:26.996776 (+ 136us) server_negotiation.cc:407] Connection header received
0806 12:57:26.996827 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:26.996828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:26.996835 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:26.996881 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:26.996987 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40598: BlockingRecv error: recv got EOF from 127.0.0.1:40598 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 12:57:26.997185 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 2022
I20260806 12:57:27.004597 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 2155
I20260806 12:57:27.009341 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 2288
I20260806 12:57:27.014516 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 1931
2026-08-06T12:57:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (595 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T12:57:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:57:27Z Disabled control of system clock
I20260806 12:57:27.041777 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:46573
--webserver_interface=127.24.83.62
--webserver_port=0
--builtin_ntp_servers=127.24.83.20:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:46573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 12:57:27.121572 2428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:27.121778 2428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:27.121819 2428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:27.122972 2428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:57:27.123013 2428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:27.123026 2428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:27.123039 2428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:57:27.123090 2428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:57:27.124431 2428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:46573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:46573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:27.124679 2428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:27.124969 2428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:27.129812 2436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:27.129792 2433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:27.129973 2428 server_base.cc:1034] running on GCE node
W20260806 12:57:27.129789 2434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:27.130213 2428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:27.130523 2428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:27.131686 2428 hybrid_clock.cc:648] HybridClock initialized: now 1786021047131665 us; error 36 us; skew 500 ppm
I20260806 12:57:27.132977 2428 webserver.cc:460] Webserver started at http://127.24.83.62:37819/ using document root <none> and password file <none>
I20260806 12:57:27.133226 2428 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:27.133272 2428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:27.133428 2428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:27.134490 2428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "9370a7c7013e43f2aaad6a0a339a9b76"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "b0d7e461c2b229d54cab25ac435f1db3"
server_key_iv: "8b540c8ae3ca7e00c775555c6cc2fa85"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.134857 2428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "9370a7c7013e43f2aaad6a0a339a9b76"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "b0d7e461c2b229d54cab25ac435f1db3"
server_key_iv: "8b540c8ae3ca7e00c775555c6cc2fa85"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.136188 2428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:57:27.136900 2442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:27.137076 2428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:27.137147 2428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "9370a7c7013e43f2aaad6a0a339a9b76"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "b0d7e461c2b229d54cab25ac435f1db3"
server_key_iv: "8b540c8ae3ca7e00c775555c6cc2fa85"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.137198 2428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:27.140377 2428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:27.140596 2428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:27.140691 2428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:27.144608 2428 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:46573
I20260806 12:57:27.144637 2494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:46573 every 8 connection(s)
I20260806 12:57:27.144954 2428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:57:27.145610 2495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:27.145677 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 2428
I20260806 12:57:27.145788 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:57:27.145975 24908 external_mini_cluster.cc:1428] Setting key 9afdce4be89803ff66810f8669753799
I20260806 12:57:27.148741 2498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:27.146522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60352 (local address 127.24.83.62:46573)
0806 12:57:27.146759 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:27.146768 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:27.146800 (+ 32us) server_negotiation.cc:407] Connection header received
0806 12:57:27.147401 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:27.147410 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:27.147464 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:27.147616 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:27.148304 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:27.148308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:27.148308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:57:27.148312 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:27.148359 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:57:27.148371 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:27.148456 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:27.148508 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 12:57:27.148563 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260806 12:57:27.149607 2495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76: Bootstrap starting.
I20260806 12:57:27.150197 2495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:27.150489 2495 log.cc:824] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:27.151216 2495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76: No bootstrap required, opened a new log
I20260806 12:57:27.152593 2495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9370a7c7013e43f2aaad6a0a339a9b76" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46573 } }
I20260806 12:57:27.152719 2495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:27.152765 2495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9370a7c7013e43f2aaad6a0a339a9b76, State: Initialized, Role: FOLLOWER
I20260806 12:57:27.152882 2495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9370a7c7013e43f2aaad6a0a339a9b76" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46573 } }
I20260806 12:57:27.152971 2495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:57:27.153031 2495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:57:27.153080 2495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:27.153597 2495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9370a7c7013e43f2aaad6a0a339a9b76" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46573 } }
I20260806 12:57:27.153726 2495 leader_election.cc:302] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9370a7c7013e43f2aaad6a0a339a9b76; no voters:
I20260806 12:57:27.153910 2495 leader_election.cc:288] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:57:27.154156 2495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:57:27.154196 2500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:27.154326 2500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [term 1 LEADER]: Becoming Leader. State: Replica: 9370a7c7013e43f2aaad6a0a339a9b76, State: Running, Role: LEADER
I20260806 12:57:27.154446 2500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9370a7c7013e43f2aaad6a0a339a9b76" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46573 } }
I20260806 12:57:27.154819 2502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9370a7c7013e43f2aaad6a0a339a9b76. Latest consensus state: current_term: 1 leader_uuid: "9370a7c7013e43f2aaad6a0a339a9b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9370a7c7013e43f2aaad6a0a339a9b76" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46573 } } }
I20260806 12:57:27.154872 2501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9370a7c7013e43f2aaad6a0a339a9b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9370a7c7013e43f2aaad6a0a339a9b76" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46573 } } }
I20260806 12:57:27.154923 2502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:27.154945 2501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:27.155157 2509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:57:27.155758 2509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:57:27.157220 2509 catalog_manager.cc:1347] Generated new cluster ID: 71316084d9324319bd85fa66d8bb5c93
I20260806 12:57:27.157271 2509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:57:27.176321 2509 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:57:27.176503 2509 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:57:27.182252 2509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9370a7c7013e43f2aaad6a0a339a9b76: Generated new TSK 0
I20260806 12:57:27.182437 2509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:57:27.184113 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46573
--builtin_ntp_servers=127.24.83.20:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 12:57:27.271467 2519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:27.271648 2519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:27.271669 2519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:27.272843 2519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:27.272882 2519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:27.272915 2519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:57:27.274466 2519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:27.274737 2519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:27.275059 2519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:27.280756 2525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:27.280776 2527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:27.280820 2524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:27.280988 2519 server_base.cc:1034] running on GCE node
I20260806 12:57:27.281265 2519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:27.281538 2519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:27.282719 2519 hybrid_clock.cc:648] HybridClock initialized: now 1786021047282704 us; error 40 us; skew 500 ppm
I20260806 12:57:27.284318 2519 webserver.cc:460] Webserver started at http://127.24.83.1:35117/ using document root <none> and password file <none>
I20260806 12:57:27.284595 2519 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:27.284662 2519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:27.284799 2519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:27.286140 2519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "3c34ad45e2024aa68cab49635582c379"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "ec122885295195a0f2946e2e1d2686a6"
server_key_iv: "7eaf66dba709b3087baef2079e37e0f5"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.286528 2519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "3c34ad45e2024aa68cab49635582c379"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "ec122885295195a0f2946e2e1d2686a6"
server_key_iv: "7eaf66dba709b3087baef2079e37e0f5"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.288100 2519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 12:57:27.288941 2534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:27.289181 2519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:27.289289 2519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "3c34ad45e2024aa68cab49635582c379"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "ec122885295195a0f2946e2e1d2686a6"
server_key_iv: "7eaf66dba709b3087baef2079e37e0f5"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.289372 2519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:27.292618 2519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:27.292891 2519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:27.293030 2519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:27.293239 2519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:27.293540 2519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:27.293592 2519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:27.293632 2519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:27.293658 2519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:27.299463 2519 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:38537
I20260806 12:57:27.299539 2648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:38537 every 8 connection(s)
I20260806 12:57:27.299865 2519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:57:27.302634 2498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:27.300612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:39397 (local address 127.24.83.62:46573)
0806 12:57:27.300654 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:27.300656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:27.300911 (+ 255us) server_negotiation.cc:407] Connection header received
0806 12:57:27.301352 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:27.301353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:27.301359 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:27.301421 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:27.302449 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:27.302450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:27.302450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:57:27.302454 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:27.302467 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:57:27.302471 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:27.302515 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:27.302570 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 12:57:27.302583 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 12:57:27.302845 2651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:27.300623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:46573 (local address 127.24.83.1:39397)
0806 12:57:27.300792 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:27.300824 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:27.301271 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:27.301471 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:27.301473 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:27.301481 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:27.302320 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:57:27.302320 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:27.302347 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:57:27.302373 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:27.302513 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:27.302516 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:27.302518 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 12:57:27.302568 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 12:57:27.302614 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260806 12:57:27.303359 2649 heartbeater.cc:347] Connected to a master server at 127.24.83.62:46573
I20260806 12:57:27.303464 2649 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:27.303613 2649 heartbeater.cc:511] Master 127.24.83.62:46573 requested a full tablet report, sending...
I20260806 12:57:27.304157 2459 ts_manager.cc:194] Registered new tserver with Master: 3c34ad45e2024aa68cab49635582c379 (127.24.83.1:38537)
I20260806 12:57:27.309490 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 2519
I20260806 12:57:27.309602 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:57:27.309800 24908 external_mini_cluster.cc:1428] Setting key c63802af037bbf8ad8be4404370cac8c
I20260806 12:57:27.310717 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46573
--builtin_ntp_servers=127.24.83.20:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 12:57:27.392424 2652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:27.392659 2652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:27.392721 2652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:27.393932 2652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:27.394002 2652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:27.394047 2652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:57:27.395370 2652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:27.395627 2652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:27.395938 2652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:27.400707 2660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:27.400755 2652 server_base.cc:1034] running on GCE node
W20260806 12:57:27.400709 2657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:27.400707 2658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:27.401084 2652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:27.401348 2652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:27.402513 2652 hybrid_clock.cc:648] HybridClock initialized: now 1786021047402494 us; error 30 us; skew 500 ppm
I20260806 12:57:27.403681 2652 webserver.cc:460] Webserver started at http://127.24.83.2:32977/ using document root <none> and password file <none>
I20260806 12:57:27.403991 2652 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:27.404059 2652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:27.404229 2652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:27.405274 2652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "aae2320fca304b8389c83a427b85a111"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "17fc9a514a2e8a29c05270ac2013a1d0"
server_key_iv: "61b7c64dbe7d26f5c7a0b4f604d04a89"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.405646 2652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "aae2320fca304b8389c83a427b85a111"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "17fc9a514a2e8a29c05270ac2013a1d0"
server_key_iv: "61b7c64dbe7d26f5c7a0b4f604d04a89"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.406934 2652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:57:27.407612 2666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:27.407784 2652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:27.407882 2652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "aae2320fca304b8389c83a427b85a111"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "17fc9a514a2e8a29c05270ac2013a1d0"
server_key_iv: "61b7c64dbe7d26f5c7a0b4f604d04a89"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.407994 2652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:27.411276 2652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:27.411532 2652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:27.411681 2652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:27.411931 2652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:27.412272 2652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:27.412317 2652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:27.412356 2652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:27.412382 2652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:27.418217 2652 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:43109
I20260806 12:57:27.418252 2780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:43109 every 8 connection(s)
I20260806 12:57:27.418550 2652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:57:27.421335 2498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:27.419250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:40117 (local address 127.24.83.62:46573)
0806 12:57:27.419307 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:27.419309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:27.419500 (+ 191us) server_negotiation.cc:407] Connection header received
0806 12:57:27.419963 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:27.419964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:27.419970 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:27.420034 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:27.421151 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:27.421152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:27.421152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:57:27.421154 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:27.421163 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:57:27.421166 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:27.421213 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:27.421271 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 12:57:27.421283 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 12:57:27.421499 2783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:27.419212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:46573 (local address 127.24.83.2:40117)
0806 12:57:27.419382 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:27.419420 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:27.419831 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:27.420087 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:27.420090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:27.420098 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:27.421029 (+ 931us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:57:27.421029 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:27.421055 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:57:27.421081 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:27.421214 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:27.421216 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:27.421218 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 12:57:27.421268 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 12:57:27.421312 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260806 12:57:27.422017 2781 heartbeater.cc:347] Connected to a master server at 127.24.83.62:46573
I20260806 12:57:27.422113 2781 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:27.422250 2781 heartbeater.cc:511] Master 127.24.83.62:46573 requested a full tablet report, sending...
I20260806 12:57:27.422621 2459 ts_manager.cc:194] Registered new tserver with Master: aae2320fca304b8389c83a427b85a111 (127.24.83.2:43109)
I20260806 12:57:27.425717 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 2652
I20260806 12:57:27.425796 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:57:27.425989 24908 external_mini_cluster.cc:1428] Setting key 3dd6b07b6004a003ea785a860a398bfa
I20260806 12:57:27.426862 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46573
--builtin_ntp_servers=127.24.83.20:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 12:57:27.512619 2784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:27.512818 2784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:27.512841 2784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:27.514097 2784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:27.514138 2784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:27.514163 2784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:57:27.515426 2784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:27.515602 2784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:27.515832 2784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:27.520591 2792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:27.520591 2790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:27.520608 2789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:27.521018 2784 server_base.cc:1034] running on GCE node
I20260806 12:57:27.521301 2784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:27.521539 2784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:27.522692 2784 hybrid_clock.cc:648] HybridClock initialized: now 1786021047522678 us; error 45 us; skew 500 ppm
I20260806 12:57:27.523867 2784 webserver.cc:460] Webserver started at http://127.24.83.3:34111/ using document root <none> and password file <none>
I20260806 12:57:27.524125 2784 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:27.524168 2784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:27.524258 2784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:27.525233 2784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "146340b6d3d94f7ba5ba2c8be7539fb5"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "67fef0436a280d5e9a5d1e66f6ba0a10"
server_key_iv: "5a268cf1aa5f547012bbeff19a48a82c"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.525519 2784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "146340b6d3d94f7ba5ba2c8be7539fb5"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "67fef0436a280d5e9a5d1e66f6ba0a10"
server_key_iv: "5a268cf1aa5f547012bbeff19a48a82c"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.526770 2784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:57:27.527468 2798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:27.527645 2784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:27.527735 2784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "146340b6d3d94f7ba5ba2c8be7539fb5"
format_stamp: "Formatted at 2026-08-06 12:57:27 on dist-test-slave-zkpd"
server_key: "67fef0436a280d5e9a5d1e66f6ba0a10"
server_key_iv: "5a268cf1aa5f547012bbeff19a48a82c"
server_key_version: "encryptionkey@0"
I20260806 12:57:27.527807 2784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:27.530998 2784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:27.531239 2784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:27.531366 2784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:27.531581 2784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:27.531876 2784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:27.531965 2784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:27.532001 2784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:27.532029 2784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:27.537662 2784 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:34725
I20260806 12:57:27.537706 2913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:34725 every 8 connection(s)
I20260806 12:57:27.537993 2784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:57:27.540916 2498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:27.538644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:40863 (local address 127.24.83.62:46573)
0806 12:57:27.538684 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:27.538687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:27.538882 (+ 195us) server_negotiation.cc:407] Connection header received
0806 12:57:27.539310 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:27.539311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:27.539315 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:27.539369 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:27.540649 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:27.540651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:27.540651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:57:27.540656 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:27.540672 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:57:27.540676 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:27.540729 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:27.540829 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 12:57:27.540844 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 12:57:27.541103 2916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:27.538590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:46573 (local address 127.24.83.3:40863)
0806 12:57:27.538760 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:27.538796 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:27.539216 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:27.539445 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:27.539448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:27.539456 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:27.540495 (+ 1039us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:57:27.540495 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:27.540522 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:57:27.540548 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:27.540726 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:27.540731 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:27.540735 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:57:27.540844 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 12:57:27.540904 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260806 12:57:27.541563 2914 heartbeater.cc:347] Connected to a master server at 127.24.83.62:46573
I20260806 12:57:27.541662 2914 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:27.541648 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 2784
I20260806 12:57:27.541738 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:57:27.541805 2914 heartbeater.cc:511] Master 127.24.83.62:46573 requested a full tablet report, sending...
I20260806 12:57:27.541934 24908 external_mini_cluster.cc:1428] Setting key 4dd4da6940022774b077344cdc90203a
I20260806 12:57:27.542155 2459 ts_manager.cc:194] Registered new tserver with Master: 146340b6d3d94f7ba5ba2c8be7539fb5 (127.24.83.3:34725)
I20260806 12:57:27.542936 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 12:57:27.544714 2498 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:57:27.544194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60362 (local address 127.24.83.62:46573)
0806 12:57:27.544229 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:27.544231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:27.544396 (+ 165us) server_negotiation.cc:407] Connection header received
0806 12:57:27.544466 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:27.544467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:27.544471 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:27.544520 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:27.544640 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60362: BlockingRecv error: recv got EOF from 127.0.0.1:60362 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 12:57:27.544836 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 2519
I20260806 12:57:27.551869 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 2652
I20260806 12:57:27.556751 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 2784
I20260806 12:57:27.561014 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 2428
2026-08-06T12:57:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (547 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:57:27 dist-test-slave-zkpd krb5kdc[2924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:57:27 dist-test-slave-zkpd krb5kdc[2924](info): set up 2 sockets
Aug 06 12:57:27 dist-test-slave-zkpd krb5kdc[2924](info): commencing operation
krb5kdc: starting...
W20260806 12:57:28.621765 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:57:28 dist-test-slave-zkpd krb5kdc[2924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:57:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:57:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:57:28.738547 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45047
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:45047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:28.818647 2937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:28.818876 2937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:28.818913 2937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:28.820053 2937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:57:28.820119 2937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:28.820142 2937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:28.820163 2937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:57:28.820204 2937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:57:28.821625 2937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:45047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:28.821902 2937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:28.822113 2937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:28.826790 2942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:28.826790 2945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:28.826908 2937 server_base.cc:1034] running on GCE node
W20260806 12:57:28.826782 2943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:28.827279 2937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:28.827570 2937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:28.828734 2937 hybrid_clock.cc:648] HybridClock initialized: now 1786021048828726 us; error 34 us; skew 500 ppm
Aug 06 12:57:28 dist-test-slave-zkpd krb5kdc[2924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021048, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:28.831297 2937 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:57:28.831765 2937 webserver.cc:460] Webserver started at http://127.24.83.62:43699/ using document root <none> and password file <none>
I20260806 12:57:28.832048 2937 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:28.832113 2937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:28.832264 2937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:28.833282 2937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "6e21d0617d9e4d64847ac26873a2e5c6"
format_stamp: "Formatted at 2026-08-06 12:57:28 on dist-test-slave-zkpd"
server_key: "a416f3478816c36f353a10132964b24a"
server_key_iv: "19d4efacd16ee8a68991c88be983ad7d"
server_key_version: "encryptionkey@0"
I20260806 12:57:28.833639 2937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "6e21d0617d9e4d64847ac26873a2e5c6"
format_stamp: "Formatted at 2026-08-06 12:57:28 on dist-test-slave-zkpd"
server_key: "a416f3478816c36f353a10132964b24a"
server_key_iv: "19d4efacd16ee8a68991c88be983ad7d"
server_key_version: "encryptionkey@0"
I20260806 12:57:28.834934 2937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:57:28.835613 2953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:28.835793 2937 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:28.835918 2937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "6e21d0617d9e4d64847ac26873a2e5c6"
format_stamp: "Formatted at 2026-08-06 12:57:28 on dist-test-slave-zkpd"
server_key: "a416f3478816c36f353a10132964b24a"
server_key_iv: "19d4efacd16ee8a68991c88be983ad7d"
server_key_version: "encryptionkey@0"
I20260806 12:57:28.836010 2937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:28.848953 2937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:28.851194 2937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:28.851365 2937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:28.855237 2937 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:45047
I20260806 12:57:28.855269 3005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:45047 every 8 connection(s)
I20260806 12:57:28.855594 2937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:57:28.856580 3006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:28.858760 3006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6: Bootstrap starting.
I20260806 12:57:28.859323 3006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:28.859611 3006 log.cc:824] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:28.860383 3006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6: No bootstrap required, opened a new log
I20260806 12:57:28.861963 3006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e21d0617d9e4d64847ac26873a2e5c6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45047 } }
I20260806 12:57:28.862092 3006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:28.862114 3006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e21d0617d9e4d64847ac26873a2e5c6, State: Initialized, Role: FOLLOWER
I20260806 12:57:28.862298 3006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e21d0617d9e4d64847ac26873a2e5c6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45047 } }
I20260806 12:57:28.862360 3006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:57:28.862401 3006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:57:28.862449 3006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:28.863070 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 2937
I20260806 12:57:28.863179 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:57:28.863338 3006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e21d0617d9e4d64847ac26873a2e5c6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45047 } }
I20260806 12:57:28.863379 24908 external_mini_cluster.cc:1428] Setting key 8e3cd96da23ce9451f103a39034e9860
I20260806 12:57:28.863425 3006 leader_election.cc:302] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e21d0617d9e4d64847ac26873a2e5c6; no voters:
I20260806 12:57:28.863626 3006 leader_election.cc:288] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:57:28.863651 3009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:28.863780 3009 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [term 1 LEADER]: Becoming Leader. State: Replica: 6e21d0617d9e4d64847ac26873a2e5c6, State: Running, Role: LEADER
I20260806 12:57:28.863852 3009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e21d0617d9e4d64847ac26873a2e5c6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45047 } }
I20260806 12:57:28.863937 3006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:57:28.864197 3009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e21d0617d9e4d64847ac26873a2e5c6. Latest consensus state: current_term: 1 leader_uuid: "6e21d0617d9e4d64847ac26873a2e5c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e21d0617d9e4d64847ac26873a2e5c6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45047 } } }
I20260806 12:57:28.864192 3010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e21d0617d9e4d64847ac26873a2e5c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e21d0617d9e4d64847ac26873a2e5c6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45047 } } }
I20260806 12:57:28.864390 3009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:28.864501 3010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6 [sys.catalog]: This master's current role is: LEADER
I20260806 12:57:28.865283 3020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:57:28.866060 3020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:57:28.868008 3020 catalog_manager.cc:1347] Generated new cluster ID: dd032167ead94cde869ed96b4f3f1dae
I20260806 12:57:28.868084 3020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:57:28 dist-test-slave-zkpd krb5kdc[2924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:57:28.875691 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:28.863845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43898 (local address 127.24.83.62:45047)
0806 12:57:28.864729 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:28.864740 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:28.864774 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:57:28.864883 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:28.864890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:28.864948 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:28.865107 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:28.865867 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:28.867086 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:28.868046 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:28.868684 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:28.868797 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:28.871615 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:28.871619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:28.871619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:28.871641 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:28.873924 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:28.874310 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:28.874314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:28.874314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:28.874448 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:28.874655 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:28.874656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:28.874656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:28.874963 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:28.875124 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:28.875198 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 12:57:28.875469 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":111,"threads_started":1}
I20260806 12:57:28.876946 3020 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:57:28.877625 3020 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:57:28.891567 3020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6: Generated new TSK 0
I20260806 12:57:28.891748 3020 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:57:28.922806 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45047
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:29.017426 3033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:29.017648 3033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:29.017686 3033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:29.018812 3033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:29.018880 3033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:29.018927 3033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:57:29.020426 3033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:29.020684 3033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:29.020917 3033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:29.025871 3038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:29.025866 3041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:29.025867 3039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:29.025962 3033 server_base.cc:1034] running on GCE node
I20260806 12:57:29.026274 3033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:29.026515 3033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:29.027678 3033 hybrid_clock.cc:648] HybridClock initialized: now 1786021049027663 us; error 26 us; skew 500 ppm
Aug 06 12:57:29 dist-test-slave-zkpd krb5kdc[2924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021049, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:29.030131 3033 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:57:29.030608 3033 webserver.cc:460] Webserver started at http://127.24.83.1:38877/ using document root <none> and password file <none>
I20260806 12:57:29.030843 3033 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:29.030916 3033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:29.031050 3033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:29.032122 3033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "11c65e2528794135897e052c3a0375a9"
format_stamp: "Formatted at 2026-08-06 12:57:29 on dist-test-slave-zkpd"
server_key: "c6afe22d413bbe99c4fb404776293ebc"
server_key_iv: "6f1b518a2226d523763fe8cfffce6bd8"
server_key_version: "encryptionkey@0"
I20260806 12:57:29.032487 3033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "11c65e2528794135897e052c3a0375a9"
format_stamp: "Formatted at 2026-08-06 12:57:29 on dist-test-slave-zkpd"
server_key: "c6afe22d413bbe99c4fb404776293ebc"
server_key_iv: "6f1b518a2226d523763fe8cfffce6bd8"
server_key_version: "encryptionkey@0"
I20260806 12:57:29.033763 3033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:57:29.034510 3048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:29.034677 3033 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:29.034775 3033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "11c65e2528794135897e052c3a0375a9"
format_stamp: "Formatted at 2026-08-06 12:57:29 on dist-test-slave-zkpd"
server_key: "c6afe22d413bbe99c4fb404776293ebc"
server_key_iv: "6f1b518a2226d523763fe8cfffce6bd8"
server_key_version: "encryptionkey@0"
I20260806 12:57:29.034857 3033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:29.049325 3033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:29.051092 3033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:29.051263 3033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:29.051498 3033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:29.051792 3033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:29.051846 3033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:29.051883 3033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:29.051939 3033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:29.057799 3033 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:33519
I20260806 12:57:29.057881 3161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:33519 every 8 connection(s)
I20260806 12:57:29.058192 3033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
Aug 06 12:57:29 dist-test-slave-zkpd krb5kdc[2924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021049, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:57:29.067639 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.058889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:40545 (local address 127.24.83.62:45047)
0806 12:57:29.058979 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:29.058982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:29.059148 (+ 166us) server_negotiation.cc:407] Connection header received
0806 12:57:29.059673 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:29.059674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:29.059690 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:29.059743 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:29.060795 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.061322 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.062341 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.062570 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.062657 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.064839 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:29.064840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:29.064840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:29.064848 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:29.066261 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:29.066655 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:29.066656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:29.066657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:29.066722 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:29.066982 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:29.066983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:29.066983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:29.067102 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:29.067180 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:29.067451 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 12:57:29.067546 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 12:57:29.067816 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.058841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45047 (local address 127.24.83.1:40545)
0806 12:57:29.059019 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:29.059068 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:29.059567 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:29.059818 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:29.059822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:29.060463 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:29.060662 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.060667 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.061438 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.061439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.062244 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.062246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.062671 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.062672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.062804 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.063238 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:29.063246 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:29.064742 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:29.066347 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:29.066354 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:29.066356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:29.066577 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:29.066802 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:29.066803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:29.066803 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:29.066913 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:29.067169 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:29.067176 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:29.067317 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 12:57:29.067430 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 12:57:29.067582 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260806 12:57:29.068104 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 3033
I20260806 12:57:29.068217 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:57:29.068446 24908 external_mini_cluster.cc:1428] Setting key ec85c8076b1194b3eed16a6d5c031496
I20260806 12:57:29.068459 3162 heartbeater.cc:347] Connected to a master server at 127.24.83.62:45047
I20260806 12:57:29.068567 3162 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:29.068763 3162 heartbeater.cc:511] Master 127.24.83.62:45047 requested a full tablet report, sending...
I20260806 12:57:29.069351 2970 ts_manager.cc:194] Registered new tserver with Master: 11c65e2528794135897e052c3a0375a9 (127.24.83.1:33519)
I20260806 12:57:29.069954 2970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:40545
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:57:29.092787 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45047
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:29.173744 3167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:29.173949 3167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:29.173971 3167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:29.175028 3167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:29.175060 3167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:29.175086 3167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:57:29.176532 3167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:29.176777 3167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:29.176970 3167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:29.181805 3174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:29.181965 3167 server_base.cc:1034] running on GCE node
W20260806 12:57:29.181872 3173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:29.181787 3176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:29.182268 3167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:29.182497 3167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:29.183657 3167 hybrid_clock.cc:648] HybridClock initialized: now 1786021049183644 us; error 43 us; skew 500 ppm
Aug 06 12:57:29 dist-test-slave-zkpd krb5kdc[2924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021049, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:29.186098 3167 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:57:29.186486 3167 webserver.cc:460] Webserver started at http://127.24.83.2:44373/ using document root <none> and password file <none>
I20260806 12:57:29.186661 3167 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:29.186702 3167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:29.186795 3167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:29.187757 3167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "7ebebc8085834d18914a26e35e3f59a1"
format_stamp: "Formatted at 2026-08-06 12:57:29 on dist-test-slave-zkpd"
server_key: "551e4bd6bec05236dd25a7c4f5b9a03a"
server_key_iv: "4c060429879d3841eeba0aed3d0854ae"
server_key_version: "encryptionkey@0"
I20260806 12:57:29.188153 3167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "7ebebc8085834d18914a26e35e3f59a1"
format_stamp: "Formatted at 2026-08-06 12:57:29 on dist-test-slave-zkpd"
server_key: "551e4bd6bec05236dd25a7c4f5b9a03a"
server_key_iv: "4c060429879d3841eeba0aed3d0854ae"
server_key_version: "encryptionkey@0"
I20260806 12:57:29.189431 3167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:57:29.190116 3183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:29.190291 3167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:57:29.190399 3167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "7ebebc8085834d18914a26e35e3f59a1"
format_stamp: "Formatted at 2026-08-06 12:57:29 on dist-test-slave-zkpd"
server_key: "551e4bd6bec05236dd25a7c4f5b9a03a"
server_key_iv: "4c060429879d3841eeba0aed3d0854ae"
server_key_version: "encryptionkey@0"
I20260806 12:57:29.190474 3167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:29.209951 3167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:29.212360 3167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:29.212520 3167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:29.212749 3167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:29.213070 3167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:29.213124 3167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:29.213166 3167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:29.213193 3167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:29.219009 3167 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:39429
I20260806 12:57:29.219108 3296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:39429 every 8 connection(s)
I20260806 12:57:29.219405 3167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:57:29 dist-test-slave-zkpd krb5kdc[2924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021049, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:57:29.227536 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 3167
I20260806 12:57:29.227643 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:57:29.227846 24908 external_mini_cluster.cc:1428] Setting key 7f3461fc94ea781cf70f8deedf938a10
I20260806 12:57:29.228228 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.220190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:33933 (local address 127.24.83.62:45047)
0806 12:57:29.220252 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:29.220255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:29.220436 (+ 181us) server_negotiation.cc:407] Connection header received
0806 12:57:29.220902 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:29.220903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:29.220912 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:29.220972 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:29.221838 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.222338 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.223262 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.223453 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.223499 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.225632 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:29.225633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:29.225633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:29.225639 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:29.226814 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:29.227163 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:29.227164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:29.227164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:29.227212 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:29.227518 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:29.227518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:29.227518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:29.227610 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:29.227691 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:29.228032 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 12:57:29.228116 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:57:29.228426 3299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.220129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45047 (local address 127.24.83.2:33933)
0806 12:57:29.220301 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:29.220344 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:29.220815 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:29.221023 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:29.221026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:29.221574 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:29.221760 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.221765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.222410 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.222411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.223183 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.223185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.223519 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.223520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.223627 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.224201 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:29.224209 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:29.225541 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:29.226885 (+ 1344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:29.226891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:29.226893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:29.227100 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:29.227267 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:29.227267 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:29.227268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:29.227443 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:29.227714 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:29.227720 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:29.227840 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 12:57:29.227998 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 12:57:29.228189 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260806 12:57:29.229022 3297 heartbeater.cc:347] Connected to a master server at 127.24.83.62:45047
I20260806 12:57:29.229121 3297 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:29.229313 3297 heartbeater.cc:511] Master 127.24.83.62:45047 requested a full tablet report, sending...
I20260806 12:57:29.229679 2970 ts_manager.cc:194] Registered new tserver with Master: 7ebebc8085834d18914a26e35e3f59a1 (127.24.83.2:39429)
I20260806 12:57:29.230190 2970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:33933
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:57:29.253975 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45047
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:57:29.336802 3302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:57:29.337051 3302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:57:29.337093 3302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:57:29.338356 3302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:57:29.338416 3302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:57:29.338461 3302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:57:29.340018 3302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:57:29.340288 3302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:57:29.340516 3302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:57:29.345361 3310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:57:29.345376 3308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:29.345561 3302 server_base.cc:1034] running on GCE node
W20260806 12:57:29.345378 3307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:57:29.345845 3302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:57:29.346122 3302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:57:29.347265 3302 hybrid_clock.cc:648] HybridClock initialized: now 1786021049347252 us; error 25 us; skew 500 ppm
Aug 06 12:57:29 dist-test-slave-zkpd krb5kdc[2924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021049, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:57:29.349802 3302 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:57:29.350255 3302 webserver.cc:460] Webserver started at http://127.24.83.3:42893/ using document root <none> and password file <none>
I20260806 12:57:29.350493 3302 fs_manager.cc:359] Metadata directory not provided
I20260806 12:57:29.350538 3302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:57:29.350682 3302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:57:29.351678 3302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "11c6b77313f8492b9cddbdb0e80a7c7a"
format_stamp: "Formatted at 2026-08-06 12:57:29 on dist-test-slave-zkpd"
server_key: "ae7f4353a66159949ac5f4302df05c4a"
server_key_iv: "32c131c2aa54cf9d1765cae5905512a2"
server_key_version: "encryptionkey@0"
I20260806 12:57:29.352062 3302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "11c6b77313f8492b9cddbdb0e80a7c7a"
format_stamp: "Formatted at 2026-08-06 12:57:29 on dist-test-slave-zkpd"
server_key: "ae7f4353a66159949ac5f4302df05c4a"
server_key_iv: "32c131c2aa54cf9d1765cae5905512a2"
server_key_version: "encryptionkey@0"
I20260806 12:57:29.353354 3302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:57:29.354084 3318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:29.354260 3302 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:57:29.354348 3302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "11c6b77313f8492b9cddbdb0e80a7c7a"
format_stamp: "Formatted at 2026-08-06 12:57:29 on dist-test-slave-zkpd"
server_key: "ae7f4353a66159949ac5f4302df05c4a"
server_key_iv: "32c131c2aa54cf9d1765cae5905512a2"
server_key_version: "encryptionkey@0"
I20260806 12:57:29.354419 3302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:57:29.367057 3302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:57:29.369352 3302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:57:29.369510 3302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:57:29.369728 3302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:57:29.370028 3302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:57:29.370061 3302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:29.370083 3302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:57:29.370127 3302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:57:29.376183 3302 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:42887
I20260806 12:57:29.376225 3431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:42887 every 8 connection(s)
I20260806 12:57:29.376518 3302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:57:29.378378 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 3302
I20260806 12:57:29.378468 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:57:29.378641 24908 external_mini_cluster.cc:1428] Setting key 845569798c4b73beb0efde1a07da7660
Aug 06 12:57:29 dist-test-slave-zkpd krb5kdc[2924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021049, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:57:29.385354 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.377255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:51785 (local address 127.24.83.62:45047)
0806 12:57:29.377300 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:29.377302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:29.377485 (+ 183us) server_negotiation.cc:407] Connection header received
0806 12:57:29.377937 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:29.377938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:29.377951 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:29.378007 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:29.379019 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.379540 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.380630 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.380843 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.380930 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.382980 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:29.382981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:29.382982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:29.382989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:29.384240 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:29.384595 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:29.384596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:29.384596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:29.384647 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:29.384863 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:29.384863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:29.384863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:29.384959 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:29.385008 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:29.385234 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 12:57:29.385294 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 12:57:29.385593 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.377214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45047 (local address 127.24.83.3:51785)
0806 12:57:29.377366 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:29.377400 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:29.377841 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:29.378055 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:29.378058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:29.378668 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:57:29.378925 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.378931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.379636 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.379637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.380525 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.380528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.380941 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.380942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.381059 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.381570 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:57:29.381578 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:57:29.382898 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:57:29.384310 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:29.384315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:29.384317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:29.384524 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:29.384705 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:57:29.384706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:57:29.384706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:57:29.384807 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:57:29.385013 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:57:29.385018 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:57:29.385123 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 12:57:29.385203 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 12:57:29.385332 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260806 12:57:29.386128 3432 heartbeater.cc:347] Connected to a master server at 127.24.83.62:45047
I20260806 12:57:29.386240 3432 heartbeater.cc:464] Registering TS with master...
I20260806 12:57:29.386415 3432 heartbeater.cc:511] Master 127.24.83.62:45047 requested a full tablet report, sending...
I20260806 12:57:29.386829 2970 ts_manager.cc:194] Registered new tserver with Master: 11c6b77313f8492b9cddbdb0e80a7c7a (127.24.83.3:42887)
I20260806 12:57:29.387208 2970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:51785
I20260806 12:57:29.390340 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:57:29.397528 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.391623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43912 (local address 127.24.83.62:45047)
0806 12:57:29.391682 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:29.391683 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:29.391920 (+ 237us) server_negotiation.cc:407] Connection header received
0806 12:57:29.392022 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:29.392023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:29.392033 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:29.392097 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:57:29.392655 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.393185 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.393908 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.394108 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.394179 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.395237 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:57:29.395238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:57:29.395239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:57:29.395246 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:57:29.396399 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:29.396731 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:29.396732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:29.396733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:29.396794 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:57:29.396989 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:57:29.396990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:57:29.396991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:57:29.397134 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:57:29.397239 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:29.397357 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 12:57:29.397434 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 12:57:29.398979 2970 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:57:29.400615 2970 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:57:29.406898 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.403049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:42887 (local address 127.0.0.1:44126)
0806 12:57:29.403264 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:29.403275 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:29.403330 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:29.403641 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:29.403641 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:29.403644 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:29.403691 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.403692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.404583 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.404584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.405788 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.405791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.406604 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.406605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.406687 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.406700 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 12:57:29.406769 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 12:57:29.406825 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":70,"threads_started":1}
I20260806 12:57:29.406921 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.403242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44126 (local address 127.24.83.3:42887)
0806 12:57:29.403439 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:29.403444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:29.403453 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:57:29.403486 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:29.403488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:29.403501 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:29.403570 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:29.403768 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.404478 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.405901 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.406505 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.406600 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.406660 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:29.406809 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 12:57:29.406850 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260806 12:57:29.407217 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.402858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:39429 (local address 127.0.0.1:49202)
0806 12:57:29.403066 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:29.403112 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:29.403176 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:29.403719 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:29.403720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:29.403722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:29.403777 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.403778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.405102 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.405103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.406139 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.406142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.407003 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.407004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.407066 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.407069 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:57:29.407098 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 12:57:29.407137 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":1,"thread_start_us":49,"threads_started":1}
I20260806 12:57:29.407217 3443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.402910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:33519 (local address 127.0.0.1:49964)
0806 12:57:29.403203 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:29.403212 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:29.403260 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:29.403530 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:29.403533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:29.403536 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:29.403603 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.403605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.404955 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.404956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.406005 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.406007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.406991 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.406992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.407062 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.407066 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:57:29.407114 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 12:57:29.407144 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":34,"thread_start_us":85,"threads_started":1}
I20260806 12:57:29.407423 3446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.403017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.24.83.1:33519)
0806 12:57:29.403210 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:29.403215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:29.403311 (+ 96us) server_negotiation.cc:407] Connection header received
0806 12:57:29.403381 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:29.403394 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:29.403412 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:29.403498 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:29.403698 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.404844 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.406092 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.406827 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.406900 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.406954 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:29.407261 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 12:57:29.407309 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260806 12:57:29.407822 3366 tablet_service.cc:1467] Processing CreateTablet for tablet 46562fedcf6f4dfca529c72318e56080 (DEFAULT_TABLE table=test-table [id=a3e854105a974e9d93eb0f5bef6cd154]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:29.408017 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.402964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.24.83.2:39429)
0806 12:57:29.403458 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:29.403462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:29.403491 (+ 29us) server_negotiation.cc:407] Connection header received
0806 12:57:29.403532 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:29.403534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:29.403618 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:29.403875 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:29.403986 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.405007 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.406224 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.406914 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.407780 (+ 866us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.407848 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:29.407875 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 12:57:29.407944 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":95,"threads_started":1}
I20260806 12:57:29.408190 3366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46562fedcf6f4dfca529c72318e56080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:29.408120 3096 tablet_service.cc:1467] Processing CreateTablet for tablet 46562fedcf6f4dfca529c72318e56080 (DEFAULT_TABLE table=test-table [id=a3e854105a974e9d93eb0f5bef6cd154]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:29.408458 3096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46562fedcf6f4dfca529c72318e56080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:29.408591 3231 tablet_service.cc:1467] Processing CreateTablet for tablet 46562fedcf6f4dfca529c72318e56080 (DEFAULT_TABLE table=test-table [id=a3e854105a974e9d93eb0f5bef6cd154]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:57:29.408877 3231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46562fedcf6f4dfca529c72318e56080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:57:29.410506 3449 tablet_bootstrap.cc:492] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a: Bootstrap starting.
I20260806 12:57:29.411197 3449 tablet_bootstrap.cc:654] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:29.411712 3449 log.cc:824] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:29.412004 3450 tablet_bootstrap.cc:492] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9: Bootstrap starting.
I20260806 12:57:29.412123 3451 tablet_bootstrap.cc:492] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: Bootstrap starting.
I20260806 12:57:29.412607 3450 tablet_bootstrap.cc:654] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:29.412684 3451 tablet_bootstrap.cc:654] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: Neither blocks nor log segments found. Creating new log.
I20260806 12:57:29.413116 3450 log.cc:824] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:29.413148 3451 log.cc:824] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: Log is configured to *not* fsync() on all Append() calls
I20260806 12:57:29.413910 3450 tablet_bootstrap.cc:492] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9: No bootstrap required, opened a new log
I20260806 12:57:29.413990 3450 ts_tablet_manager.cc:1397] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 12:57:29.414129 3451 tablet_bootstrap.cc:492] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: No bootstrap required, opened a new log
I20260806 12:57:29.414196 3451 ts_tablet_manager.cc:1397] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:57:29.416010 3450 raft_consensus.cc:348] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ebebc8085834d18914a26e35e3f59a1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39429 } } peers { permanent_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 42887 } } peers { permanent_uuid: "11c65e2528794135897e052c3a0375a9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33519 } }
I20260806 12:57:29.416189 3450 raft_consensus.cc:374] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:29.416116 3451 raft_consensus.cc:348] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ebebc8085834d18914a26e35e3f59a1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39429 } } peers { permanent_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 42887 } } peers { permanent_uuid: "11c65e2528794135897e052c3a0375a9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33519 } }
I20260806 12:57:29.416224 3449 tablet_bootstrap.cc:492] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a: No bootstrap required, opened a new log
I20260806 12:57:29.416225 3450 raft_consensus.cc:729] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c65e2528794135897e052c3a0375a9, State: Initialized, Role: FOLLOWER
I20260806 12:57:29.416240 3451 raft_consensus.cc:374] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:29.416273 3451 raft_consensus.cc:729] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ebebc8085834d18914a26e35e3f59a1, State: Initialized, Role: FOLLOWER
I20260806 12:57:29.416316 3449 ts_tablet_manager.cc:1397] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260806 12:57:29.416363 3450 consensus_queue.cc:260] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ebebc8085834d18914a26e35e3f59a1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39429 } } peers { permanent_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 42887 } } peers { permanent_uuid: "11c65e2528794135897e052c3a0375a9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33519 } }
I20260806 12:57:29.416416 3451 consensus_queue.cc:260] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ebebc8085834d18914a26e35e3f59a1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39429 } } peers { permanent_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 42887 } } peers { permanent_uuid: "11c65e2528794135897e052c3a0375a9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33519 } }
I20260806 12:57:29.416697 3451 ts_tablet_manager.cc:1428] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 12:57:29.416882 3450 ts_tablet_manager.cc:1428] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 12:57:29.416903 3297 heartbeater.cc:503] Master 127.24.83.62:45047 was elected leader, sending a full tablet report...
I20260806 12:57:29.417057 3162 heartbeater.cc:503] Master 127.24.83.62:45047 was elected leader, sending a full tablet report...
I20260806 12:57:29.418212 3449 raft_consensus.cc:348] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ebebc8085834d18914a26e35e3f59a1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39429 } } peers { permanent_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 42887 } } peers { permanent_uuid: "11c65e2528794135897e052c3a0375a9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33519 } }
I20260806 12:57:29.418329 3449 raft_consensus.cc:374] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:57:29.418349 3449 raft_consensus.cc:729] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c6b77313f8492b9cddbdb0e80a7c7a, State: Initialized, Role: FOLLOWER
I20260806 12:57:29.418495 3449 consensus_queue.cc:260] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ebebc8085834d18914a26e35e3f59a1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39429 } } peers { permanent_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 42887 } } peers { permanent_uuid: "11c65e2528794135897e052c3a0375a9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33519 } }
I20260806 12:57:29.418759 3449 ts_tablet_manager.cc:1428] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 12:57:29.418805 3432 heartbeater.cc:503] Master 127.24.83.62:45047 was elected leader, sending a full tablet report...
W20260806 12:57:29.469957 3298 tablet.cc:2369] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:57:29.555606 3457 raft_consensus.cc:482] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:57:29.555724 3457 raft_consensus.cc:504] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ebebc8085834d18914a26e35e3f59a1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39429 } } peers { permanent_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 42887 } } peers { permanent_uuid: "11c65e2528794135897e052c3a0375a9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33519 } }
I20260806 12:57:29.556208 3457 leader_election.cc:288] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11c6b77313f8492b9cddbdb0e80a7c7a (127.24.83.3:42887), 11c65e2528794135897e052c3a0375a9 (127.24.83.1:33519)
W20260806 12:57:29.559036 3163 tablet.cc:2369] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:57:29.559132 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.556448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:55649 (local address 127.24.83.3:42887)
0806 12:57:29.556492 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:29.556494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:29.556502 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:57:29.556550 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:29.556551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:29.556564 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:29.556638 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:29.556818 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.557352 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.558397 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.558773 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.558865 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.558876 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:29.559022 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 12:57:29.559072 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 12:57:29.559132 3299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.556354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:42887 (local address 127.24.83.2:55649)
0806 12:57:29.556426 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:29.556439 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:29.556488 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:29.556640 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:29.556642 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:29.556643 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:29.556724 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.556726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.557432 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.557433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.558307 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.558309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.558865 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.558866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.558946 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.558950 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:57:29.559000 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 12:57:29.559040 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 12:57:29.559238 3446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.556455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:58505 (local address 127.24.83.1:33519)
0806 12:57:29.556504 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:29.556506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:29.556663 (+ 157us) server_negotiation.cc:407] Connection header received
0806 12:57:29.556715 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:29.556716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:29.556729 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:29.556817 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:57:29.556986 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.557501 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.558498 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.558890 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.558962 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.558971 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:29.559117 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 12:57:29.559166 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 12:57:29.559432 3459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.556374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:33519 (local address 127.24.83.2:58505)
0806 12:57:29.556575 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:29.556583 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 12:57:29.556659 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:57:29.556825 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:57:29.556826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:57:29.556827 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:57:29.556910 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.556912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.557596 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.557597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.558412 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:57:29.558415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.558959 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.558960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:57:29.559027 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.559030 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:57:29.559321 (+ 291us) client_negotiation.cc:240] Negotiation successful
0806 12:57:29.559366 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":121,"threads_started":1}
I20260806 12:57:29.559522 3386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46562fedcf6f4dfca529c72318e56080" candidate_uuid: "7ebebc8085834d18914a26e35e3f59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" is_pre_election: true
I20260806 12:57:29.559695 3386 raft_consensus.cc:2454] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ebebc8085834d18914a26e35e3f59a1 in term 0.
I20260806 12:57:29.559746 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46562fedcf6f4dfca529c72318e56080" candidate_uuid: "7ebebc8085834d18914a26e35e3f59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c65e2528794135897e052c3a0375a9" is_pre_election: true
I20260806 12:57:29.559966 3116 raft_consensus.cc:2454] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ebebc8085834d18914a26e35e3f59a1 in term 0.
I20260806 12:57:29.559979 3184 leader_election.cc:302] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c6b77313f8492b9cddbdb0e80a7c7a, 7ebebc8085834d18914a26e35e3f59a1; no voters:
I20260806 12:57:29.560112 3457 raft_consensus.cc:2781] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:57:29.560169 3457 raft_consensus.cc:482] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:57:29.560195 3457 raft_consensus.cc:3037] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:29.560920 3457 raft_consensus.cc:504] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ebebc8085834d18914a26e35e3f59a1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39429 } } peers { permanent_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 42887 } } peers { permanent_uuid: "11c65e2528794135897e052c3a0375a9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33519 } }
I20260806 12:57:29.561060 3457 leader_election.cc:288] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [CANDIDATE]: Term 1 election: Requested vote from peers 11c6b77313f8492b9cddbdb0e80a7c7a (127.24.83.3:42887), 11c65e2528794135897e052c3a0375a9 (127.24.83.1:33519)
I20260806 12:57:29.561251 3386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46562fedcf6f4dfca529c72318e56080" candidate_uuid: "7ebebc8085834d18914a26e35e3f59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a"
I20260806 12:57:29.561260 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46562fedcf6f4dfca529c72318e56080" candidate_uuid: "7ebebc8085834d18914a26e35e3f59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c65e2528794135897e052c3a0375a9"
I20260806 12:57:29.561322 3116 raft_consensus.cc:3037] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:29.561344 3386 raft_consensus.cc:3037] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:57:29.562048 3386 raft_consensus.cc:2454] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ebebc8085834d18914a26e35e3f59a1 in term 1.
I20260806 12:57:29.562080 3116 raft_consensus.cc:2454] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ebebc8085834d18914a26e35e3f59a1 in term 1.
I20260806 12:57:29.562247 3184 leader_election.cc:302] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c6b77313f8492b9cddbdb0e80a7c7a, 7ebebc8085834d18914a26e35e3f59a1; no voters:
I20260806 12:57:29.562335 3457 raft_consensus.cc:2781] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:57:29.562510 3457 raft_consensus.cc:686] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 1 LEADER]: Becoming Leader. State: Replica: 7ebebc8085834d18914a26e35e3f59a1, State: Running, Role: LEADER
I20260806 12:57:29.562606 3457 consensus_queue.cc:237] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ebebc8085834d18914a26e35e3f59a1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39429 } } peers { permanent_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 42887 } } peers { permanent_uuid: "11c65e2528794135897e052c3a0375a9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33519 } }
I20260806 12:57:29.563436 2969 catalog_manager.cc:5495] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ebebc8085834d18914a26e35e3f59a1 (127.24.83.2). New cstate: current_term: 1 leader_uuid: "7ebebc8085834d18914a26e35e3f59a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ebebc8085834d18914a26e35e3f59a1" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 42887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11c65e2528794135897e052c3a0375a9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33519 } health_report { overall_health: UNKNOWN } } }
I20260806 12:57:29.589406 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.586801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49204 (local address 127.24.83.2:39429)
0806 12:57:29.586923 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 12:57:29.586926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:57:29.586935 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:57:29.587045 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:57:29.587046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:57:29.587079 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:57:29.587154 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:57:29.587316 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.587796 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.588603 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:57:29.588800 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:57:29.588880 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:57:29.589067 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:57:29.589159 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:57:29.589259 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 12:57:29.589277 (+ 18us) server_negotiation.cc:299] Negotiation successful
0806 12:57:29.589295 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 12:57:29.591239 3386 raft_consensus.cc:1261] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a [term 1 FOLLOWER]: Refusing update from remote peer 7ebebc8085834d18914a26e35e3f59a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:29.591253 3116 raft_consensus.cc:1261] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9 [term 1 FOLLOWER]: Refusing update from remote peer 7ebebc8085834d18914a26e35e3f59a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:57:29.591602 3460 consensus_queue.cc:1035] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11c65e2528794135897e052c3a0375a9" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 33519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:29.591732 3457 consensus_queue.cc:1035] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11c6b77313f8492b9cddbdb0e80a7c7a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 42887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:57:29.595153 3466 mvcc.cc:205] Tried to move back new op lower bound from 7315542219123011584 to 7315542219008962560. Current Snapshot: MvccSnapshot[applied={T|T < 7315542219123011584}]
I20260806 12:57:29.595678 3464 mvcc.cc:205] Tried to move back new op lower bound from 7315542219123011584 to 7315542219008962560. Current Snapshot: MvccSnapshot[applied={T|T < 7315542219123011584 or (T in {7315542219123011584})}]
I20260806 12:57:29.597172 2969 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:29.597272 2969 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:57:29.598069 2969 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e21d0617d9e4d64847ac26873a2e5c6: Sending DeleteTablet for 3 replicas of tablet 46562fedcf6f4dfca529c72318e56080
I20260806 12:57:29.598409 3096 tablet_service.cc:1514] Processing DeleteTablet for tablet 46562fedcf6f4dfca529c72318e56080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:29 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:49964
I20260806 12:57:29.598443 3231 tablet_service.cc:1514] Processing DeleteTablet for tablet 46562fedcf6f4dfca529c72318e56080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:29 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:49202
I20260806 12:57:29.598709 3474 tablet_replica.cc:330] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9: stopping tablet replica
I20260806 12:57:29.598708 3473 tablet_replica.cc:330] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: stopping tablet replica
I20260806 12:57:29.598706 3366 tablet_service.cc:1514] Processing DeleteTablet for tablet 46562fedcf6f4dfca529c72318e56080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:57:29 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:44126
I20260806 12:57:29.598809 3473 raft_consensus.cc:2229] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:57:29.598846 3474 raft_consensus.cc:2229] T 46562fedcf6f4dfca529c72318e56080 P 11c65e2528794135897e052c3a0375a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:29.598953 3473 raft_consensus.cc:2258] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:29.599046 3475 tablet_replica.cc:330] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a: stopping tablet replica
I20260806 12:57:29.599112 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 3033
I20260806 12:57:29.600116 3473 ts_tablet_manager.cc:1905] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:29.601979 3473 ts_tablet_manager.cc:1918] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:29.602046 3473 log.cc:1196] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/46562fedcf6f4dfca529c72318e56080
I20260806 12:57:29.602316 3473 ts_tablet_manager.cc:1939] T 46562fedcf6f4dfca529c72318e56080 P 7ebebc8085834d18914a26e35e3f59a1: Deleting consensus metadata
I20260806 12:57:29.602840 2954 catalog_manager.cc:4841] TS 7ebebc8085834d18914a26e35e3f59a1 (127.24.83.2:39429): tablet 46562fedcf6f4dfca529c72318e56080 (table test-table [id=a3e854105a974e9d93eb0f5bef6cd154]) successfully deleted
I20260806 12:57:29.604425 3475 raft_consensus.cc:2229] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:57:29.604570 3475 raft_consensus.cc:2258] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:57:29.605034 3475 ts_tablet_manager.cc:1905] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:57:29.606976 3475 ts_tablet_manager.cc:1918] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:57:29.607054 3475 log.cc:1196] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/46562fedcf6f4dfca529c72318e56080
I20260806 12:57:29.607322 3475 ts_tablet_manager.cc:1939] T 46562fedcf6f4dfca529c72318e56080 P 11c6b77313f8492b9cddbdb0e80a7c7a: Deleting consensus metadata
I20260806 12:57:29.607949 2954 catalog_manager.cc:4841] TS 11c6b77313f8492b9cddbdb0e80a7c7a (127.24.83.3:42887): tablet 46562fedcf6f4dfca529c72318e56080 (table test-table [id=a3e854105a974e9d93eb0f5bef6cd154]) successfully deleted
W20260806 12:57:29.608445 2955 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:33519 (error 108)
I20260806 12:57:29.609004 3443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:57:29.608864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:33519 (local address 127.0.0.1:49966)
0806 12:57:29.608925 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 12:57:29.608968 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:33519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 12:57:29.609027 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 3167
W20260806 12:57:29.611011 2955 catalog_manager.cc:4584] TS 11c65e2528794135897e052c3a0375a9 (127.24.83.1:33519): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 46562fedcf6f4dfca529c72318e56080: Network error: Client connection negotiation failed: client connection to 127.24.83.1:33519: connect: Connection refused (error 111)
I20260806 12:57:29.614140 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 3302
I20260806 12:57:29.619191 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 2937
2026-08-06T12:57:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2057 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:57:29 dist-test-slave-zkpd krb5kdc[3479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:57:29 dist-test-slave-zkpd krb5kdc[3479](info): set up 2 sockets
Aug 06 12:57:29 dist-test-slave-zkpd krb5kdc[3479](info): commencing operation
krb5kdc: starting...
W20260806 12:57:30.708721 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:57:30 dist-test-slave-zkpd krb5kdc[3479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:57:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:57:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.83.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.83.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/rangeradmin_127.24.83.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/rangeradmin_127.24.83.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.83.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.83.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/rangerlookup_127.24.83.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/rangerlookup_127.24.83.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.83.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.83.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/HTTP_127.24.83.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/HTTP_127.24.83.20@KRBTEST.COM.keytab.
I20260806 12:57:30.866451 24908 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskzNDPsb/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/postgres -l logfile start
2026-08-06 12:57:33.267 UTC [3509] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 12:57:33.268 UTC [3509] LOG: listening on IPv4 address "127.24.83.20", port 50069
2026-08-06 12:57:33.271 UTC [3509] LOG: listening on Unix socket "/tmp/.s.PGSQL.50069"
2026-08-06 12:57:33.306 UTC [3512] LOG: database system was shut down at 2026-08-06 12:57:31 UTC
2026-08-06 12:57:33.309 UTC [3509] LOG: database system is ready to accept connections
I20260806 12:57:34.273281 24908 mini_postgres.cc:96] Postgres bound to 50069
2026-08-06 12:57:34.280 UTC [3520] FATAL: database "slave" does not exist
127.24.83.20:50069 - accepting connections
I20260806 12:57:34.281419 24908 mini_ranger.cc:161] Starting Ranger...
I20260806 12:57:34.297103 24908 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 12:57:34.736914 24908 mini_ranger.cc:87] Created ranger Postgres database
I20260806 12:57:34.737001 24908 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-admin
2026-08-06 12:57:35,064 [I] DB FLAVOR :POSTGRES
2026-08-06 12:57:35,064 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 12:57:35,064 [I] Checking connection..
2026-08-06 12:57:35,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 12:57:35,299 [I] Checking connection passed.
2026-08-06 12:57:35,299 [I] --------- Verifying version history table ---------
2026-08-06 12:57:35,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:57:35,533 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 12:57:35,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:57:35,766 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 12:57:35,766 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 12:57:35,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 12:57:36.018 UTC [3584] WARNING: there is no transaction in progress
2026-08-06 12:57:36,024 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 12:57:36,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:57:36,282 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 12:57:36,282 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 12:57:36,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 12:57:36,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 12:57:36,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zkpd.c.gcp-upstream.internal','N') ;"
2026-08-06 12:57:37,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 12:57:37,228 [I] Table x_portal_user does not exist in database ranger
2026-08-06 12:57:37,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 12:57:37,466 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 12:57:37,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 12:57:37,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 12:57:37,954 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 12:57:37,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 12:57:38.878 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:38.893 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:38.908 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:38.921 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:38.938 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:38.957 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:39.010 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:39.016 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:39.030 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:39.039 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:39.048 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:39.061 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:39.070 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:39.076 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:39.082 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:39.088 UTC [3694] WARNING: there is no transaction in progress
2026-08-06 12:57:39,686 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 12:57:39,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 12:57:39,911 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 12:57:39,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 12:57:40,163 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 12:57:40,164 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 12:57:40,417 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 12:57:40,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 12:57:40,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 12:57:40,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zkpd.c.gcp-upstream.internal';"
2026-08-06 12:57:41,147 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 12:57:41,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 12:57:41,402 [I] DB_PATCHES have already been applied
I20260806 12:57:41.408288 24908 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-admin:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskzNDPsb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 12:57:41.408351 24908 mini_ranger.cc:193] Using host: 127.24.83.20
I20260806 12:57:41.411592 24908 mini_ranger.cc:227] Ranger admin URL: http://127.24.83.20:60915
Aug 06, 2026 12:57:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:57:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:57:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 12:57:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 12:57:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 12:57:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 12:57:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 12:57:42.543222 24908 mini_ranger.cc:160] Time spent starting Ranger: real 8.262s user 0.005s sys 0.005s
Aug 06, 2026 12:58:04 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 12:58:04 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 12:58:04 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 12:58:04 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 12:58:05 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 12:58:05.714123 24908 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.83.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.83.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/rangerkms_127.24.83.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/rangerkms_127.24.83.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.83.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.83.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.83.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/HTTP_127.24.83.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.83.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/HTTP_127.24.83.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 12:58:05 dist-test-slave-zkpd krb5kdc[3479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021085, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 12:58:05.818732 24908 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 12:58:05.830191 24908 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 12:58:06.645081 24908 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 12:58:06.645164 24908 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-kms
I20260806 12:58:06.645187 24908 mini_ranger_kms.cc:227] Using postgres at 127.24.83.20:50069
2026-08-06 12:58:07,259 [I] DB FLAVOR :POSTGRES
2026-08-06 12:58:07,259 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 12:58:07,259 [I] Checking connection
2026-08-06 12:58:07,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 12:58:07,606 [I] connection success
2026-08-06 12:58:07,606 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 12:58:07,606 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 12:58:07,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 12:58:07,860 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 12:58:07,860 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 12:58:07,860 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 12:58:07,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:50069/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 12:58:08,123 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 12:58:08.644155 24908 mini_ranger_kms.cc:310] Created kms service
I20260806 12:58:08.859367 24908 mini_ranger_kms.cc:326] Created kudu user
I20260806 12:58:08.904786 24908 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 12:58:08 dist-test-slave-zkpd krb5kdc[3479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021085, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:58:09.300526 24908 mini_ranger_kms.cc:384] Added ranger policy
I20260806 12:58:09.300655 24908 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-kms:/tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 12:58:09.300679 24908 mini_ranger_kms.cc:240] Using host: 127.24.83.20
I20260806 12:58:09.304756 24908 mini_ranger_kms.cc:276] Ranger KMS PID: 3878
I20260806 12:58:09.304870 24908 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.83.20:54215
Aug 06, 2026 12:58:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:58:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:58:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 12:58:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 12:58:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 12:58:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 12:58:11.592900 24908 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.774s user 0.000s sys 0.585s
I20260806 12:58:11.593015 24908 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 12:58:11.593088 24908 mini_ranger_kms.cc:401] 127.24.83.20:54215/kms/v1/keys
Aug 06 12:58:12 dist-test-slave-zkpd krb5kdc[3479](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786021092, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.83.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 12:58:12 dist-test-slave-zkpd krb5kdc[3479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021092, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.83.20@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
Aug 06 12:58:12 dist-test-slave-zkpd krb5kdc[3479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021092, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.83.20@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
Aug 06 12:58:15 dist-test-slave-zkpd krb5kdc[3479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:58:15.363473 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33385
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:33385
--ranger_config_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:54215/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:58:15.460809 3928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:58:15.461112 3928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:58:15.461149 3928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:58:15.463272 3928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:58:15.463336 3928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:58:15.463356 3928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:58:15.463373 3928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:58:15.463440 3928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:58:15.466188 3928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.83.20:54215/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:33385
--ranger_config_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:58:15.466640 3928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:58:15.466918 3928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:58:15.472671 3936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:58:15.472904 3933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:58:15.472657 3934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:58:15.472862 3928 server_base.cc:1034] running on GCE node
I20260806 12:58:15.474269 3928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:58:15.474572 3928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:58:15.475761 3928 hybrid_clock.cc:648] HybridClock initialized: now 1786021095475737 us; error 41 us; skew 500 ppm
Aug 06 12:58:15 dist-test-slave-zkpd krb5kdc[3479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021095, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:58:15.481679 3928 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:58:15.482221 3928 webserver.cc:460] Webserver started at http://127.24.83.62:32959/ using document root <none> and password file <none>
I20260806 12:58:15.482482 3928 fs_manager.cc:359] Metadata directory not provided
I20260806 12:58:15.482544 3928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:58:15.482699 3928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:58:15 dist-test-slave-zkpd krb5kdc[3479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021095, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:58:15.550210 3928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "9da7229e77324ee4a95502d43569d60f"
format_stamp: "Formatted at 2026-08-06 12:58:15 on dist-test-slave-zkpd"
server_key: "68524ce6ae8f6fbef932840c6ed280e1"
server_key_iv: "d33912c95e1a42059ac7e6ff8cb7a949"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:15.550714 3928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "9da7229e77324ee4a95502d43569d60f"
format_stamp: "Formatted at 2026-08-06 12:58:15 on dist-test-slave-zkpd"
server_key: "68524ce6ae8f6fbef932840c6ed280e1"
server_key_iv: "d33912c95e1a42059ac7e6ff8cb7a949"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:15.552414 3928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260806 12:58:15.568274 3943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:58:15.568677 3928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:58:15.568789 3928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "9da7229e77324ee4a95502d43569d60f"
format_stamp: "Formatted at 2026-08-06 12:58:15 on dist-test-slave-zkpd"
server_key: "68524ce6ae8f6fbef932840c6ed280e1"
server_key_iv: "d33912c95e1a42059ac7e6ff8cb7a949"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:15.568885 3928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:58:15.584034 3928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:58:15.585968 3928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:58:15.586171 3928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:58:15.595011 3928 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:33385
I20260806 12:58:15.595235 3996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:33385 every 8 connection(s)
I20260806 12:58:15.595485 3928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:58:15.596590 3997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:58:15.600003 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 3928
I20260806 12:58:15.600131 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:58:15.600339 24908 external_mini_cluster.cc:1428] Setting key 427866cc84a54594d318ae2644f8aacb
I20260806 12:58:15.600582 3997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f: Bootstrap starting.
I20260806 12:58:15.601989 3997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f: Neither blocks nor log segments found. Creating new log.
I20260806 12:58:15.602607 3997 log.cc:824] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:58:15.603585 3997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f: No bootstrap required, opened a new log
Aug 06 12:58:15 dist-test-slave-zkpd krb5kdc[3479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:58:15.607287 3997 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da7229e77324ee4a95502d43569d60f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33385 } }
I20260806 12:58:15.607501 3997 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:58:15.607586 3997 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9da7229e77324ee4a95502d43569d60f, State: Initialized, Role: FOLLOWER
I20260806 12:58:15.607764 3997 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da7229e77324ee4a95502d43569d60f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33385 } }
I20260806 12:58:15.607869 3997 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:58:15.607950 3997 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:58:15.608011 3997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:58:15.609037 3997 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da7229e77324ee4a95502d43569d60f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33385 } }
I20260806 12:58:15.609159 3997 leader_election.cc:302] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9da7229e77324ee4a95502d43569d60f; no voters:
I20260806 12:58:15.609344 3997 leader_election.cc:288] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:58:15.609494 4002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:58:15.609730 3997 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:58:15.609774 4002 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [term 1 LEADER]: Becoming Leader. State: Replica: 9da7229e77324ee4a95502d43569d60f, State: Running, Role: LEADER
I20260806 12:58:15.609977 4002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da7229e77324ee4a95502d43569d60f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33385 } }
I20260806 12:58:15.612179 4000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:15.600891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.24.83.62:33385)
0806 12:58:15.601177 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 12:58:15.601188 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:58:15.601239 (+ 51us) server_negotiation.cc:407] Connection header received
0806 12:58:15.601813 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:58:15.601820 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:58:15.601867 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:58:15.602152 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:58:15.602841 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:15.604115 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:15.604897 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:15.605360 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:15.605496 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:15.608094 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:58:15.608098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:58:15.608098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:58:15.608110 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:58:15.610251 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:58:15.610665 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:58:15.610667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:58:15.610668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:58:15.610777 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:58:15.611077 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:58:15.611078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:58:15.611078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:58:15.611424 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:58:15.611600 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 12:58:15.611793 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 12:58:15.611938 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":153,"threads_started":1}
I20260806 12:58:15.612511 4004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9da7229e77324ee4a95502d43569d60f. Latest consensus state: current_term: 1 leader_uuid: "9da7229e77324ee4a95502d43569d60f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da7229e77324ee4a95502d43569d60f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33385 } } }
I20260806 12:58:15.612609 4004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [sys.catalog]: This master's current role is: LEADER
I20260806 12:58:15.612684 3997 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 12:58:15.612776 4003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9da7229e77324ee4a95502d43569d60f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da7229e77324ee4a95502d43569d60f" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33385 } } }
I20260806 12:58:15.612831 4003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f [sys.catalog]: This master's current role is: LEADER
I20260806 12:58:15.613296 4016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:58:15.617973 4016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:58:15.626075 4016 catalog_manager.cc:1347] Generated new cluster ID: a36acb54b1bd49a6b175f9419385d387
I20260806 12:58:15.626189 4016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:58:15.635180 4016 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:58:15.636053 4016 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:58:15.642056 4016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f: Generated new TSK 0
I20260806 12:58:15.642318 4016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:58:16.587404 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:54215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33385
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:58:16.752278 4042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:58:16.752611 4042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:58:16.752660 4042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:58:16.754659 4042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:58:16.754783 4042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:58:16.754827 4042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:58:16.757285 4042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.83.20:54215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:58:16.757562 4042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:58:16.758016 4042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:58:16.764381 4048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:58:16.764343 4050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:58:16.764400 4047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:58:16.765117 4042 server_base.cc:1034] running on GCE node
I20260806 12:58:16.765337 4042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:58:16.765722 4042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:58:16.766916 4042 hybrid_clock.cc:648] HybridClock initialized: now 1786021096766884 us; error 42 us; skew 500 ppm
Aug 06 12:58:16 dist-test-slave-zkpd krb5kdc[3479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021096, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:58:16.770829 4042 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:58:16.771344 4042 webserver.cc:460] Webserver started at http://127.24.83.1:38189/ using document root <none> and password file <none>
I20260806 12:58:16.771620 4042 fs_manager.cc:359] Metadata directory not provided
I20260806 12:58:16.771668 4042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:58:16.771795 4042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:58:16 dist-test-slave-zkpd krb5kdc[3479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021096, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:58:16.786559 4042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "07649c0aa82347b7b638dfc68c124fbc"
format_stamp: "Formatted at 2026-08-06 12:58:16 on dist-test-slave-zkpd"
server_key: "4acdf19e1e50a32772056290184def79"
server_key_iv: "2c8b0df6f9c18c8d99e6f50630abea7d"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:16.787843 4042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "07649c0aa82347b7b638dfc68c124fbc"
format_stamp: "Formatted at 2026-08-06 12:58:16 on dist-test-slave-zkpd"
server_key: "4acdf19e1e50a32772056290184def79"
server_key_iv: "2c8b0df6f9c18c8d99e6f50630abea7d"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:16.790217 4042 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:58:16.804122 4057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:58:16.804405 4042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:58:16.804587 4042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "07649c0aa82347b7b638dfc68c124fbc"
format_stamp: "Formatted at 2026-08-06 12:58:16 on dist-test-slave-zkpd"
server_key: "4acdf19e1e50a32772056290184def79"
server_key_iv: "2c8b0df6f9c18c8d99e6f50630abea7d"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:16.804817 4042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:58:16.815565 4042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:58:16.818280 4042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:58:16.818533 4042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:58:16.819031 4042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:58:16.819578 4042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:58:16.819638 4042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:58:16.819710 4042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:58:16.819741 4042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:58:16.836961 4042 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:38747
I20260806 12:58:16.837411 4042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:58:16.840379 4171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:38747 every 8 connection(s)
I20260806 12:58:16.844748 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 4042
I20260806 12:58:16.844856 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:58:16.845064 24908 external_mini_cluster.cc:1428] Setting key 60e7dbb4347a890d582f48ba3267c553
Aug 06 12:58:16 dist-test-slave-zkpd krb5kdc[3479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021096, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:58:16.852087 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:16.838428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:35629 (local address 127.24.83.62:33385)
0806 12:58:16.838555 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:58:16.838558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:58:16.838848 (+ 290us) server_negotiation.cc:407] Connection header received
0806 12:58:16.839638 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:58:16.839639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:58:16.839654 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:58:16.839730 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:58:16.840892 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:16.841437 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:16.842642 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:16.842868 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:16.842961 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:16.848742 (+ 5781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:58:16.848744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:58:16.848744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:58:16.848752 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:58:16.850495 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:58:16.850961 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:58:16.850963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:58:16.850963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:58:16.851029 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:58:16.851306 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:58:16.851307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:58:16.851307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:58:16.851432 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:58:16.851513 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 12:58:16.851881 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 12:58:16.851990 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":49,"threads_started":1}
I20260806 12:58:16.852308 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:16.838422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33385 (local address 127.24.83.1:35629)
0806 12:58:16.838689 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 12:58:16.838733 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 12:58:16.839508 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:58:16.839853 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:58:16.839857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:58:16.840361 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:58:16.840781 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:16.840788 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:16.841542 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:16.841543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:16.842520 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:16.842522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:16.842974 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:16.842975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:16.843131 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:16.843801 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:58:16.843812 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:58:16.848630 (+ 4818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:58:16.850593 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:58:16.850601 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:58:16.850603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:58:16.850876 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:58:16.851111 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:58:16.851112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:58:16.851112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:58:16.851232 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:58:16.851519 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:58:16.851527 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:58:16.851733 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 12:58:16.851858 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 12:58:16.852034 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":146,"threads_started":1}
I20260806 12:58:16.853011 4172 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33385
I20260806 12:58:16.853145 4172 heartbeater.cc:464] Registering TS with master...
I20260806 12:58:16.853333 4172 heartbeater.cc:511] Master 127.24.83.62:33385 requested a full tablet report, sending...
I20260806 12:58:16.853910 3961 ts_manager.cc:194] Registered new tserver with Master: 07649c0aa82347b7b638dfc68c124fbc (127.24.83.1:38747)
I20260806 12:58:16.854525 3961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:35629
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:58:16.888028 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:54215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33385
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:58:17.030119 4180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:58:17.030390 4180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:58:17.030426 4180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:58:17.032168 4180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:58:17.032223 4180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:58:17.032260 4180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:58:17.034585 4180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.83.20:54215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:58:17.034847 4180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:58:17.035156 4180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:58:17.043248 4187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:58:17.044373 4188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:58:17.044649 4180 server_base.cc:1034] running on GCE node
W20260806 12:58:17.044788 4190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:58:17.045243 4180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:58:17.045598 4180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:58:17.046783 4180 hybrid_clock.cc:648] HybridClock initialized: now 1786021097046745 us; error 51 us; skew 500 ppm
Aug 06 12:58:17 dist-test-slave-zkpd krb5kdc[3479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021097, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:58:17.051290 4180 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:58:17.051859 4180 webserver.cc:460] Webserver started at http://127.24.83.2:44049/ using document root <none> and password file <none>
I20260806 12:58:17.052222 4180 fs_manager.cc:359] Metadata directory not provided
I20260806 12:58:17.052296 4180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:58:17.052431 4180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:58:17 dist-test-slave-zkpd krb5kdc[3479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021097, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:58:17.070928 4180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "3857ab01f8bd418685e535c66ab49329"
format_stamp: "Formatted at 2026-08-06 12:58:17 on dist-test-slave-zkpd"
server_key: "ccbebd376d35087feafc64dcc5823979"
server_key_iv: "c2ff1c1e7704da48856a7e17102d41e1"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:17.071521 4180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "3857ab01f8bd418685e535c66ab49329"
format_stamp: "Formatted at 2026-08-06 12:58:17 on dist-test-slave-zkpd"
server_key: "ccbebd376d35087feafc64dcc5823979"
server_key_iv: "c2ff1c1e7704da48856a7e17102d41e1"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:17.073948 4180 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:58:17.086583 4197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:58:17.086745 4180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:58:17.086825 4180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "3857ab01f8bd418685e535c66ab49329"
format_stamp: "Formatted at 2026-08-06 12:58:17 on dist-test-slave-zkpd"
server_key: "ccbebd376d35087feafc64dcc5823979"
server_key_iv: "c2ff1c1e7704da48856a7e17102d41e1"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:17.086901 4180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 06 12:58:17 dist-test-slave-zkpd krb5kdc[3479](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786021097, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:58:17.128611 4180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:58:17.131436 4180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:58:17.131686 4180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:58:17.132004 4180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:58:17.132468 4180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:58:17.132562 4180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:58:17.132650 4180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:58:17.132710 4180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:58:17.151803 4180 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:33123
I20260806 12:58:17.152405 4180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:58:17.153185 4311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:33123 every 8 connection(s)
I20260806 12:58:17.159469 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 4180
I20260806 12:58:17.159591 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:58:17.159787 24908 external_mini_cluster.cc:1428] Setting key e694971d471f2255c0d64ef6efa81353
Aug 06 12:58:17 dist-test-slave-zkpd krb5kdc[3479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021097, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:58:17.167371 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.154882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:55255 (local address 127.24.83.62:33385)
0806 12:58:17.154961 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 12:58:17.154963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:58:17.155610 (+ 647us) server_negotiation.cc:407] Connection header received
0806 12:58:17.156730 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:58:17.156732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:58:17.156745 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:58:17.156777 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:58:17.157916 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.158696 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.160586 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.160865 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.160915 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.163999 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:58:17.164000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:58:17.164001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:58:17.164008 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:58:17.165712 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:58:17.166218 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:58:17.166219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:58:17.166220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:58:17.166282 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:58:17.166550 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:58:17.166551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:58:17.166551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:58:17.166681 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:58:17.166731 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:58:17.167152 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 12:58:17.167255 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 12:58:17.168190 4314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.154468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33385 (local address 127.24.83.2:55255)
0806 12:58:17.155448 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0806 12:58:17.155499 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 12:58:17.156592 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:58:17.156974 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:58:17.156979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:58:17.157358 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:58:17.157793 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.157801 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.158826 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.158827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.160493 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.160495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.161015 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.161016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.161235 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.161869 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:58:17.161880 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:58:17.163851 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:58:17.165806 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:58:17.165814 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:58:17.165817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:58:17.166105 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:58:17.166351 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:58:17.166352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:58:17.166353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:58:17.166479 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:58:17.166765 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:58:17.166770 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:58:17.166993 (+ 223us) client_negotiation.cc:769] Sending connection context
0806 12:58:17.167721 (+ 728us) client_negotiation.cc:240] Negotiation successful
0806 12:58:17.167886 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"thread_start_us":185,"threads_started":1}
I20260806 12:58:17.168920 4312 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33385
I20260806 12:58:17.169016 4312 heartbeater.cc:464] Registering TS with master...
I20260806 12:58:17.169217 4312 heartbeater.cc:511] Master 127.24.83.62:33385 requested a full tablet report, sending...
I20260806 12:58:17.169651 3961 ts_manager.cc:194] Registered new tserver with Master: 3857ab01f8bd418685e535c66ab49329 (127.24.83.2:33123)
I20260806 12:58:17.170120 3961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:55255
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:58:17.195152 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:54215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33385
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:58:17.361665 4317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:58:17.361966 4317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:58:17.362021 4317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:58:17.364414 4317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:58:17.364480 4317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:58:17.364521 4317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:58:17.366988 4317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.83.20:54215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:58:17.367308 4317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:58:17.367645 4317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:58:17.380210 4326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:58:17.380277 4323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:58:17.381623 4317 server_base.cc:1034] running on GCE node
W20260806 12:58:17.384061 4324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:58:17.384439 4317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:58:17.384760 4317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:58:17.385946 4317 hybrid_clock.cc:648] HybridClock initialized: now 1786021097385909 us; error 49 us; skew 500 ppm
Aug 06 12:58:17 dist-test-slave-zkpd krb5kdc[3479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021097, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:58:17.390455 4317 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:58:17.391183 4317 webserver.cc:460] Webserver started at http://127.24.83.3:38681/ using document root <none> and password file <none>
I20260806 12:58:17.391511 4317 fs_manager.cc:359] Metadata directory not provided
I20260806 12:58:17.391644 4317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:58:17.391877 4317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:58:17 dist-test-slave-zkpd krb5kdc[3479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021097, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:58:17.414670 4317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "53f257f9ebf44e72a264db96c9a27f3b"
format_stamp: "Formatted at 2026-08-06 12:58:17 on dist-test-slave-zkpd"
server_key: "4f69231f074c9ea60da94c99962fd06f"
server_key_iv: "28622fae9847443f8ee0adad13390059"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:17.415917 4317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "53f257f9ebf44e72a264db96c9a27f3b"
format_stamp: "Formatted at 2026-08-06 12:58:17 on dist-test-slave-zkpd"
server_key: "4f69231f074c9ea60da94c99962fd06f"
server_key_iv: "28622fae9847443f8ee0adad13390059"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:17.419653 4317 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:58:17.439198 4333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:58:17.439488 4317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 12:58:17.439595 4317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "53f257f9ebf44e72a264db96c9a27f3b"
format_stamp: "Formatted at 2026-08-06 12:58:17 on dist-test-slave-zkpd"
server_key: "4f69231f074c9ea60da94c99962fd06f"
server_key_iv: "28622fae9847443f8ee0adad13390059"
server_key_version: "kuduclusterkey@0"
I20260806 12:58:17.439694 4317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:58:17.460460 4317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:58:17.464294 4317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:58:17.464608 4317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:58:17.465066 4317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:58:17.465740 4317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:58:17.465842 4317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:58:17.465919 4317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:58:17.465956 4317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:58:17.483013 4317 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:36137
I20260806 12:58:17.483701 4317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:58:17.485180 4446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:36137 every 8 connection(s)
I20260806 12:58:17.493891 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 4317
I20260806 12:58:17.494004 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:58:17.494189 24908 external_mini_cluster.cc:1428] Setting key 654309352d66b48c278366b3bc05fa45
Aug 06 12:58:17 dist-test-slave-zkpd krb5kdc[3479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021097, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:58:17.506026 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.488709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:51029 (local address 127.24.83.62:33385)
0806 12:58:17.488769 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 12:58:17.488771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:58:17.490080 (+ 1309us) server_negotiation.cc:407] Connection header received
0806 12:58:17.491265 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:58:17.491266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:58:17.491280 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:58:17.491312 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:58:17.492531 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.493339 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.496530 (+ 3191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.496821 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.496863 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.502261 (+ 5398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:58:17.502263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:58:17.502263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:58:17.502271 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:58:17.503930 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:58:17.504385 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:58:17.504386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:58:17.504387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:58:17.504441 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:58:17.504689 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:58:17.504690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:58:17.504690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:58:17.504823 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:58:17.504872 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 12:58:17.505856 (+ 984us) server_negotiation.cc:299] Negotiation successful
0806 12:58:17.505935 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:58:17.506807 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.488715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33385 (local address 127.24.83.3:51029)
0806 12:58:17.489905 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0806 12:58:17.489959 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 12:58:17.491119 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:58:17.491542 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:58:17.491546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:58:17.492024 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:58:17.492404 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.492412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.493459 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.493460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.496425 (+ 2965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.496428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.496953 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.496954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.497158 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.498919 (+ 1761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:58:17.498929 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:58:17.502147 (+ 3218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:58:17.504022 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:58:17.504029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:58:17.504032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:58:17.504300 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:58:17.504503 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:58:17.504503 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:58:17.504504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:58:17.504622 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:58:17.504904 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:58:17.504910 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:58:17.505673 (+ 763us) client_negotiation.cc:769] Sending connection context
0806 12:58:17.506369 (+ 696us) client_negotiation.cc:240] Negotiation successful
0806 12:58:17.506526 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1124,"thread_start_us":200,"threads_started":1}
I20260806 12:58:17.507486 4447 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33385
I20260806 12:58:17.507644 4447 heartbeater.cc:464] Registering TS with master...
I20260806 12:58:17.507833 4447 heartbeater.cc:511] Master 127.24.83.62:33385 requested a full tablet report, sending...
I20260806 12:58:17.508247 3961 ts_manager.cc:194] Registered new tserver with Master: 53f257f9ebf44e72a264db96c9a27f3b (127.24.83.3:36137)
I20260806 12:58:17.508695 3961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:51029
I20260806 12:58:17.516485 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:58:17.526922 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.518112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45282 (local address 127.24.83.62:33385)
0806 12:58:17.518290 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 12:58:17.518294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:58:17.518530 (+ 236us) server_negotiation.cc:407] Connection header received
0806 12:58:17.518573 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:58:17.518574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:58:17.518594 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:58:17.518701 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:58:17.519486 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.520452 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.521550 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.521939 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.522040 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.523632 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:58:17.523634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:58:17.523634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:58:17.523643 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:58:17.525513 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:58:17.525940 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:58:17.525941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:58:17.525941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:58:17.526014 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:58:17.526252 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:58:17.526253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:58:17.526253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:58:17.526437 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:58:17.526592 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 12:58:17.526703 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 12:58:17.526807 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 12:58:17.528978 3961 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:58:17.532099 3961 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:58:17.544569 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.536114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40368 (local address 127.24.83.3:36137)
0806 12:58:17.539260 (+ 3146us) server_negotiation.cc:206] Beginning negotiation
0806 12:58:17.539266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:58:17.539275 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:58:17.539343 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:58:17.539346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:58:17.539361 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:58:17.539440 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:58:17.539835 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.541086 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.543028 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.543876 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.543967 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.544126 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 12:58:17.544386 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 12:58:17.544428 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3100,"thread_start_us":142,"threads_started":1}
I20260806 12:58:17.544601 4461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.535612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:36137 (local address 127.0.0.1:40368)
0806 12:58:17.539062 (+ 3450us) negotiation.cc:107] Waiting for socket to connect
0806 12:58:17.539071 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:58:17.539108 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:58:17.539570 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:58:17.539574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:58:17.539590 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:58:17.539726 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.539728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.541253 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.541254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.542733 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.542736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.544052 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.544053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.544211 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.544245 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 12:58:17.544354 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 12:58:17.544498 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3428,"thread_start_us":83,"threads_started":1}
I20260806 12:58:17.546012 4381 tablet_service.cc:1467] Processing CreateTablet for tablet 3fa860d5159c4856930463954aa6c267 (DEFAULT_TABLE table=test-table [id=593466313cd44095901247ec92248376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:58:17.546496 4381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fa860d5159c4856930463954aa6c267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:58:17.551424 4466 tablet_bootstrap.cc:492] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b: Bootstrap starting.
I20260806 12:58:17.554255 4466 tablet_bootstrap.cc:654] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b: Neither blocks nor log segments found. Creating new log.
I20260806 12:58:17.555416 4466 log.cc:824] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b: Log is configured to *not* fsync() on all Append() calls
I20260806 12:58:17.556834 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.535352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:38747 (local address 127.0.0.1:49842)
0806 12:58:17.538776 (+ 3424us) negotiation.cc:107] Waiting for socket to connect
0806 12:58:17.538822 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 12:58:17.538887 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:58:17.542881 (+ 3994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:58:17.542881 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:58:17.542884 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:58:17.542924 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.542926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.554492 (+ 11566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.554493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.555418 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.555422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.556615 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.556617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.556679 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.556684 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:58:17.556714 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 12:58:17.556762 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3348,"thread_start_us":101,"threads_started":1}
I20260806 12:58:17.557468 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.536466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49842 (local address 127.24.83.1:38747)
0806 12:58:17.541675 (+ 5209us) server_negotiation.cc:206] Beginning negotiation
0806 12:58:17.541679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:58:17.541688 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:58:17.541720 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:58:17.541722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:58:17.541737 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:58:17.544186 (+ 2449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:58:17.544260 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.552390 (+ 8130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.555579 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.556512 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.557200 (+ 688us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.557272 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:58:17.557299 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 12:58:17.557345 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5174,"thread_start_us":57,"threads_started":1}
I20260806 12:58:17.558120 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.536112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55334 (local address 127.24.83.2:33123)
0806 12:58:17.541166 (+ 5054us) server_negotiation.cc:206] Beginning negotiation
0806 12:58:17.541177 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:58:17.541187 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:58:17.541274 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:58:17.541276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:58:17.541294 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:58:17.541374 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:58:17.541833 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.552784 (+ 10951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.554144 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.557050 (+ 2906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.557197 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.557406 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 12:58:17.557920 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 12:58:17.557964 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3640,"thread_start_us":110,"threads_started":1}
I20260806 12:58:17.558461 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.535835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:33123 (local address 127.0.0.1:55334)
0806 12:58:17.539597 (+ 3762us) negotiation.cc:107] Waiting for socket to connect
0806 12:58:17.539607 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:58:17.539649 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:58:17.541507 (+ 1858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:58:17.541508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:58:17.541510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:58:17.541552 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.541554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.552868 (+ 11314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.552869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.553959 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.553962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.557734 (+ 3772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.557735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.557805 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.557810 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:58:17.558352 (+ 542us) client_negotiation.cc:240] Negotiation successful
0806 12:58:17.558397 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3738,"thread_start_us":98,"threads_started":1}
I20260806 12:58:17.558540 4466 tablet_bootstrap.cc:492] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b: No bootstrap required, opened a new log
I20260806 12:58:17.558638 4466 ts_tablet_manager.cc:1397] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 12:58:17.559413 4245 tablet_service.cc:1467] Processing CreateTablet for tablet 3fa860d5159c4856930463954aa6c267 (DEFAULT_TABLE table=test-table [id=593466313cd44095901247ec92248376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:58:17.559854 4245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fa860d5159c4856930463954aa6c267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:58:17.563755 4466 raft_consensus.cc:348] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07649c0aa82347b7b638dfc68c124fbc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38747 } } peers { permanent_uuid: "53f257f9ebf44e72a264db96c9a27f3b" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 36137 } } peers { permanent_uuid: "3857ab01f8bd418685e535c66ab49329" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33123 } }
I20260806 12:58:17.564553 4466 raft_consensus.cc:374] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:58:17.564723 4466 raft_consensus.cc:729] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f257f9ebf44e72a264db96c9a27f3b, State: Initialized, Role: FOLLOWER
I20260806 12:58:17.565095 4466 consensus_queue.cc:260] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07649c0aa82347b7b638dfc68c124fbc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38747 } } peers { permanent_uuid: "53f257f9ebf44e72a264db96c9a27f3b" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 36137 } } peers { permanent_uuid: "3857ab01f8bd418685e535c66ab49329" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33123 } }
I20260806 12:58:17.565797 4466 ts_tablet_manager.cc:1428] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b: Time spent starting tablet: real 0.007s user 0.001s sys 0.006s
I20260806 12:58:17.566093 4447 heartbeater.cc:503] Master 127.24.83.62:33385 was elected leader, sending a full tablet report...
I20260806 12:58:17.566139 4106 tablet_service.cc:1467] Processing CreateTablet for tablet 3fa860d5159c4856930463954aa6c267 (DEFAULT_TABLE table=test-table [id=593466313cd44095901247ec92248376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:58:17.566498 4106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fa860d5159c4856930463954aa6c267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:58:17.566807 4468 tablet_bootstrap.cc:492] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329: Bootstrap starting.
I20260806 12:58:17.569798 4468 tablet_bootstrap.cc:654] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329: Neither blocks nor log segments found. Creating new log.
I20260806 12:58:17.570680 4468 log.cc:824] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329: Log is configured to *not* fsync() on all Append() calls
I20260806 12:58:17.572482 4471 tablet_bootstrap.cc:492] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc: Bootstrap starting.
I20260806 12:58:17.572754 4468 tablet_bootstrap.cc:492] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329: No bootstrap required, opened a new log
I20260806 12:58:17.572856 4468 ts_tablet_manager.cc:1397] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 12:58:17.574158 4471 tablet_bootstrap.cc:654] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc: Neither blocks nor log segments found. Creating new log.
I20260806 12:58:17.575043 4471 log.cc:824] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc: Log is configured to *not* fsync() on all Append() calls
I20260806 12:58:17.576478 4471 tablet_bootstrap.cc:492] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc: No bootstrap required, opened a new log
I20260806 12:58:17.576551 4471 ts_tablet_manager.cc:1397] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 12:58:17.576532 4468 raft_consensus.cc:348] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07649c0aa82347b7b638dfc68c124fbc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38747 } } peers { permanent_uuid: "53f257f9ebf44e72a264db96c9a27f3b" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 36137 } } peers { permanent_uuid: "3857ab01f8bd418685e535c66ab49329" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33123 } }
I20260806 12:58:17.576749 4468 raft_consensus.cc:374] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:58:17.576792 4468 raft_consensus.cc:729] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3857ab01f8bd418685e535c66ab49329, State: Initialized, Role: FOLLOWER
I20260806 12:58:17.576944 4468 consensus_queue.cc:260] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07649c0aa82347b7b638dfc68c124fbc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38747 } } peers { permanent_uuid: "53f257f9ebf44e72a264db96c9a27f3b" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 36137 } } peers { permanent_uuid: "3857ab01f8bd418685e535c66ab49329" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33123 } }
I20260806 12:58:17.577404 4468 ts_tablet_manager.cc:1428] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 12:58:17.580081 4312 heartbeater.cc:503] Master 127.24.83.62:33385 was elected leader, sending a full tablet report...
I20260806 12:58:17.583324 4471 raft_consensus.cc:348] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07649c0aa82347b7b638dfc68c124fbc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38747 } } peers { permanent_uuid: "53f257f9ebf44e72a264db96c9a27f3b" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 36137 } } peers { permanent_uuid: "3857ab01f8bd418685e535c66ab49329" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33123 } }
I20260806 12:58:17.583628 4471 raft_consensus.cc:374] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:58:17.583737 4471 raft_consensus.cc:729] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07649c0aa82347b7b638dfc68c124fbc, State: Initialized, Role: FOLLOWER
I20260806 12:58:17.583956 4471 consensus_queue.cc:260] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07649c0aa82347b7b638dfc68c124fbc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38747 } } peers { permanent_uuid: "53f257f9ebf44e72a264db96c9a27f3b" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 36137 } } peers { permanent_uuid: "3857ab01f8bd418685e535c66ab49329" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33123 } }
I20260806 12:58:17.584338 4471 ts_tablet_manager.cc:1428] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 12:58:17.587955 4172 heartbeater.cc:503] Master 127.24.83.62:33385 was elected leader, sending a full tablet report...
W20260806 12:58:17.591145 4173 tablet.cc:2369] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:58:17.616477 4469 raft_consensus.cc:482] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:58:17.616632 4469 raft_consensus.cc:504] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07649c0aa82347b7b638dfc68c124fbc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38747 } } peers { permanent_uuid: "53f257f9ebf44e72a264db96c9a27f3b" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 36137 } } peers { permanent_uuid: "3857ab01f8bd418685e535c66ab49329" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33123 } }
I20260806 12:58:17.617197 4469 leader_election.cc:288] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07649c0aa82347b7b638dfc68c124fbc (127.24.83.1:38747), 3857ab01f8bd418685e535c66ab49329 (127.24.83.2:33123)
I20260806 12:58:17.621871 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.617353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:33123 (local address 127.24.83.3:52217)
0806 12:58:17.617481 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 12:58:17.617494 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:58:17.617542 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:58:17.617895 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:58:17.617896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:58:17.617898 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:58:17.618036 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.618038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.619198 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.619199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.620498 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.620501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.621513 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.621514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.621665 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.621670 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:58:17.621753 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 12:58:17.621796 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260806 12:58:17.621882 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.617651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:52217 (local address 127.24.83.2:33123)
0806 12:58:17.617699 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 12:58:17.617701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:58:17.617760 (+ 59us) server_negotiation.cc:407] Connection header received
0806 12:58:17.617789 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:58:17.617790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:58:17.617803 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:58:17.617863 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:58:17.618183 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.618974 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.620808 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.621431 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.621512 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.621524 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 12:58:17.621772 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 12:58:17.621813 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 12:58:17.622296 4251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa860d5159c4856930463954aa6c267" candidate_uuid: "53f257f9ebf44e72a264db96c9a27f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3857ab01f8bd418685e535c66ab49329" is_pre_election: true
I20260806 12:58:17.622490 4251 raft_consensus.cc:2454] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53f257f9ebf44e72a264db96c9a27f3b in term 0.
I20260806 12:58:17.622758 4335 leader_election.cc:302] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3857ab01f8bd418685e535c66ab49329, 53f257f9ebf44e72a264db96c9a27f3b; no voters:
I20260806 12:58:17.622888 4469 raft_consensus.cc:2781] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:58:17.622931 4469 raft_consensus.cc:482] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:58:17.622954 4469 raft_consensus.cc:3037] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:58:17.624336 4469 raft_consensus.cc:504] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07649c0aa82347b7b638dfc68c124fbc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38747 } } peers { permanent_uuid: "53f257f9ebf44e72a264db96c9a27f3b" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 36137 } } peers { permanent_uuid: "3857ab01f8bd418685e535c66ab49329" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33123 } }
I20260806 12:58:17.624491 4469 leader_election.cc:288] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [CANDIDATE]: Term 1 election: Requested vote from peers 07649c0aa82347b7b638dfc68c124fbc (127.24.83.1:38747), 3857ab01f8bd418685e535c66ab49329 (127.24.83.2:33123)
I20260806 12:58:17.624944 4251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa860d5159c4856930463954aa6c267" candidate_uuid: "53f257f9ebf44e72a264db96c9a27f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3857ab01f8bd418685e535c66ab49329"
I20260806 12:58:17.625066 4251 raft_consensus.cc:3037] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:58:17.626499 4251 raft_consensus.cc:2454] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53f257f9ebf44e72a264db96c9a27f3b in term 1.
I20260806 12:58:17.626715 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.617709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:41453 (local address 127.24.83.1:38747)
0806 12:58:17.617759 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 12:58:17.617761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:58:17.618165 (+ 404us) server_negotiation.cc:407] Connection header received
0806 12:58:17.619452 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:58:17.619454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:58:17.619467 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:58:17.619497 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:58:17.620006 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.620725 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.625438 (+ 4713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.626179 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.626232 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.626242 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:58:17.626596 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 12:58:17.626644 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 12:58:17.626952 4335 leader_election.cc:302] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3857ab01f8bd418685e535c66ab49329, 53f257f9ebf44e72a264db96c9a27f3b; no voters:
I20260806 12:58:17.627019 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.617357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:38747 (local address 127.24.83.3:41453)
0806 12:58:17.618077 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 12:58:17.618116 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:58:17.619348 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:58:17.619562 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:58:17.619562 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:58:17.619564 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:58:17.619881 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.619883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.621584 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.621584 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.625300 (+ 3716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:58:17.625304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.626372 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.626373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:58:17.626496 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.626501 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:58:17.626828 (+ 327us) client_negotiation.cc:240] Negotiation successful
0806 12:58:17.626872 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":13,"thread_start_us":145,"threads_started":1}
I20260806 12:58:17.627363 4469 raft_consensus.cc:2781] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:58:17.627553 4469 raft_consensus.cc:686] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 1 LEADER]: Becoming Leader. State: Replica: 53f257f9ebf44e72a264db96c9a27f3b, State: Running, Role: LEADER
I20260806 12:58:17.627494 4113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa860d5159c4856930463954aa6c267" candidate_uuid: "53f257f9ebf44e72a264db96c9a27f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07649c0aa82347b7b638dfc68c124fbc" is_pre_election: true
I20260806 12:58:17.627682 4469 consensus_queue.cc:237] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07649c0aa82347b7b638dfc68c124fbc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38747 } } peers { permanent_uuid: "53f257f9ebf44e72a264db96c9a27f3b" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 36137 } } peers { permanent_uuid: "3857ab01f8bd418685e535c66ab49329" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33123 } }
I20260806 12:58:17.627729 4113 raft_consensus.cc:2454] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53f257f9ebf44e72a264db96c9a27f3b in term 0.
I20260806 12:58:17.627933 4115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa860d5159c4856930463954aa6c267" candidate_uuid: "53f257f9ebf44e72a264db96c9a27f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07649c0aa82347b7b638dfc68c124fbc"
I20260806 12:58:17.628037 4115 raft_consensus.cc:3037] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:58:17.629243 4115 raft_consensus.cc:2454] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53f257f9ebf44e72a264db96c9a27f3b in term 1.
I20260806 12:58:17.629956 3961 catalog_manager.cc:5495] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 53f257f9ebf44e72a264db96c9a27f3b (127.24.83.3). New cstate: current_term: 1 leader_uuid: "53f257f9ebf44e72a264db96c9a27f3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07649c0aa82347b7b638dfc68c124fbc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53f257f9ebf44e72a264db96c9a27f3b" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 36137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3857ab01f8bd418685e535c66ab49329" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33123 } health_report { overall_health: UNKNOWN } } }
I20260806 12:58:17.640753 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.637153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40370 (local address 127.24.83.3:36137)
0806 12:58:17.637198 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 12:58:17.637200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:58:17.637208 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:58:17.637244 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:58:17.637245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:58:17.637264 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:58:17.637337 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:58:17.637570 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.638370 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.639712 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:58:17.640064 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:58:17.640145 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:58:17.640352 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:58:17.640483 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:58:17.640577 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 12:58:17.640647 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 12:58:17.640668 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:58:17.643581 4115 raft_consensus.cc:1261] T 3fa860d5159c4856930463954aa6c267 P 07649c0aa82347b7b638dfc68c124fbc [term 1 FOLLOWER]: Refusing update from remote peer 53f257f9ebf44e72a264db96c9a27f3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:58:17.643597 4251 raft_consensus.cc:1261] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329 [term 1 FOLLOWER]: Refusing update from remote peer 53f257f9ebf44e72a264db96c9a27f3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:58:17.644104 4477 consensus_queue.cc:1035] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "07649c0aa82347b7b638dfc68c124fbc" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 38747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:58:17.644232 4469 consensus_queue.cc:1035] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3857ab01f8bd418685e535c66ab49329" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:58:17.648541 4480 mvcc.cc:205] Tried to move back new op lower bound from 7315542415944085504 to 7315542415883968512. Current Snapshot: MvccSnapshot[applied={T|T < 7315542415944085504}]
I20260806 12:58:17.652047 4482 mvcc.cc:205] Tried to move back new op lower bound from 7315542415944085504 to 7315542415883968512. Current Snapshot: MvccSnapshot[applied={T|T < 7315542415944085504}]
I20260806 12:58:17.652215 3961 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:58:17.652326 3961 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:58:17.653311 3961 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9da7229e77324ee4a95502d43569d60f: Sending DeleteTablet for 3 replicas of tablet 3fa860d5159c4856930463954aa6c267
I20260806 12:58:17.654109 4106 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fa860d5159c4856930463954aa6c267 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:58:17 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:49842
I20260806 12:58:17.654469 4381 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fa860d5159c4856930463954aa6c267 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:58:17 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:40368
I20260806 12:58:17.654692 4245 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fa860d5159c4856930463954aa6c267 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:58:17 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:55334
I20260806 12:58:17.655438 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 4042
W20260806 12:58:17.656750 4313 tablet.cc:2369] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:58:17.660249 4481 mvcc.cc:205] Tried to move back new op lower bound from 7315542415944085504 to 7315542415883968512. Current Snapshot: MvccSnapshot[applied={T|T < 7315542415944085504}]
I20260806 12:58:17.661103 4489 tablet_replica.cc:330] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b: stopping tablet replica
I20260806 12:58:17.661257 4489 raft_consensus.cc:2229] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:58:17.661363 4489 raft_consensus.cc:2258] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:58:17.664347 4489 ts_tablet_manager.cc:1905] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:58:17.664601 4490 tablet_replica.cc:330] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329: stopping tablet replica
I20260806 12:58:17.667567 4489 ts_tablet_manager.cc:1918] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:58:17.667778 4490 raft_consensus.cc:2229] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:58:17.667991 4490 raft_consensus.cc:2258] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 12:58:17.670621 3944 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:38747 (error 108)
I20260806 12:58:17.670663 4489 log.cc:1196] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/3fa860d5159c4856930463954aa6c267
I20260806 12:58:17.671033 4489 ts_tablet_manager.cc:1939] T 3fa860d5159c4856930463954aa6c267 P 53f257f9ebf44e72a264db96c9a27f3b: Deleting consensus metadata
I20260806 12:58:17.671497 4490 ts_tablet_manager.cc:1905] T 3fa860d5159c4856930463954aa6c267 P 3857ab01f8bd418685e535c66ab49329: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:58:17.671499 3948 catalog_manager.cc:4841] TS 53f257f9ebf44e72a264db96c9a27f3b (127.24.83.3:36137): tablet 3fa860d5159c4856930463954aa6c267 (table test-table [id=593466313cd44095901247ec92248376]) successfully deleted
I20260806 12:58:17.672039 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 4180
I20260806 12:58:17.680477 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.680332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:38747 (local address 127.0.0.1:49854)
0806 12:58:17.680391 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 12:58:17.680440 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:38747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 12:58:17.680744 3944 catalog_manager.cc:4584] TS 07649c0aa82347b7b638dfc68c124fbc (127.24.83.1:38747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3fa860d5159c4856930463954aa6c267: Network error: Client connection negotiation failed: client connection to 127.24.83.1:38747: connect: Connection refused (error 111)
I20260806 12:58:17.681421 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 4317
I20260806 12:58:17.689584 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:58:17.689470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:33123 (local address 127.0.0.1:55344)
0806 12:58:17.689521 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 12:58:17.689549 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.2:33123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 12:58:17.693009 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 3928
I20260806 12:58:17.700950 24908 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 12:58:18.044973 24908 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 12:58:18.045071 24908 mini_ranger.cc:66] Stopping Ranger...
I20260806 12:58:18.391175 24908 mini_ranger.cc:68] Stopped Ranger
2026-08-06 12:58:18.391 UTC [3509] LOG: received smart shutdown request
2026-08-06 12:58:18.395 UTC [3509] LOG: background worker "logical replication launcher" (PID 3518) exited with exit code 1
2026-08-06 12:58:18.395 UTC [3513] LOG: shutting down
2026-08-06 12:58:18.559 UTC [3509] LOG: database system is shut down
2026-08-06T12:58:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49049 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:58:18 dist-test-slave-zkpd krb5kdc[4501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:58:18 dist-test-slave-zkpd krb5kdc[4501](info): set up 2 sockets
Aug 06 12:58:18 dist-test-slave-zkpd krb5kdc[4501](info): commencing operation
krb5kdc: starting...
W20260806 12:58:19.734769 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:58:19 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:58:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:58:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.83.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.83.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/rangeradmin_127.24.83.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/rangeradmin_127.24.83.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.83.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.83.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/rangerlookup_127.24.83.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/rangerlookup_127.24.83.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.83.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.83.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/HTTP_127.24.83.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/HTTP_127.24.83.20@KRBTEST.COM.keytab.
I20260806 12:58:19.902424 24908 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskzNDPsb/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/postgres -l logfile start
2026-08-06 12:58:22.383 UTC [4531] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 12:58:22.383 UTC [4531] LOG: listening on IPv4 address "127.24.83.20", port 36203
2026-08-06 12:58:22.387 UTC [4531] LOG: listening on Unix socket "/tmp/.s.PGSQL.36203"
2026-08-06 12:58:22.422 UTC [4534] LOG: database system was shut down at 2026-08-06 12:58:20 UTC
2026-08-06 12:58:22.426 UTC [4531] LOG: database system is ready to accept connections
I20260806 12:58:23.380319 24908 mini_postgres.cc:96] Postgres bound to 36203
2026-08-06 12:58:23.386 UTC [4542] FATAL: database "slave" does not exist
127.24.83.20:36203 - accepting connections
I20260806 12:58:23.387179 24908 mini_ranger.cc:161] Starting Ranger...
I20260806 12:58:23.403206 24908 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 12:58:23.873934 24908 mini_ranger.cc:87] Created ranger Postgres database
I20260806 12:58:23.874006 24908 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-admin
2026-08-06 12:58:24,154 [I] DB FLAVOR :POSTGRES
2026-08-06 12:58:24,155 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 12:58:24,155 [I] Checking connection..
2026-08-06 12:58:24,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 12:58:24,396 [I] Checking connection passed.
2026-08-06 12:58:24,396 [I] --------- Verifying version history table ---------
2026-08-06 12:58:24,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:58:24,640 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 12:58:24,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:58:24,878 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 12:58:24,878 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 12:58:24,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 12:58:25.135 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 12:58:25,142 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 12:58:25,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:58:25,385 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 12:58:25,385 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 12:58:25,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 12:58:25,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 12:58:25,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zkpd.c.gcp-upstream.internal','N') ;"
2026-08-06 12:58:26,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 12:58:26,413 [I] Table x_portal_user does not exist in database ranger
2026-08-06 12:58:26,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 12:58:26,684 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 12:58:26,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 12:58:26,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 12:58:27,173 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 12:58:27,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 12:58:28.108 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.123 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.139 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.155 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.170 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.186 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.239 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.246 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.259 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.270 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.280 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.295 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.305 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.312 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.318 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28.325 UTC [4716] WARNING: there is no transaction in progress
2026-08-06 12:58:28,940 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 12:58:28,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 12:58:29,192 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 12:58:29,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 12:58:29,423 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 12:58:29,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 12:58:29,655 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 12:58:29,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 12:58:29,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 12:58:30,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zkpd.c.gcp-upstream.internal';"
2026-08-06 12:58:30,371 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 12:58:30,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 12:58:30,646 [I] DB_PATCHES have already been applied
I20260806 12:58:30.651893 24908 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-admin:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskzNDPsb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 12:58:30.652041 24908 mini_ranger.cc:193] Using host: 127.24.83.20
I20260806 12:58:30.655320 24908 mini_ranger.cc:227] Ranger admin URL: http://127.24.83.20:60971
Aug 06, 2026 12:58:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:58:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:58:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 12:58:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 12:58:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 12:58:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 12:58:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 12:58:31.774276 24908 mini_ranger.cc:160] Time spent starting Ranger: real 8.387s user 0.002s sys 0.007s
Aug 06, 2026 12:58:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 12:58:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 06, 2026 12:58:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 12:58:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 12:58:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 12:58:53.295255 24908 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.83.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.83.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/rangerkms_127.24.83.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.83.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/rangerkms_127.24.83.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.83.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.83.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.83.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/HTTP_127.24.83.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.83.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/HTTP_127.24.83.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 12:58:53 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021133, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 12:58:53.379793 24908 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 12:58:53.388808 24908 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 12:58:54.229264 24908 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 12:58:54.229349 24908 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-kms
I20260806 12:58:54.229368 24908 mini_ranger_kms.cc:227] Using postgres at 127.24.83.20:36203
2026-08-06 12:58:54,408 [I] DB FLAVOR :POSTGRES
2026-08-06 12:58:54,408 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 12:58:54,408 [I] Checking connection
2026-08-06 12:58:54,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 12:58:54,679 [I] connection success
2026-08-06 12:58:54,679 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 12:58:54,679 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 12:58:54,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 12:58:54,992 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 12:58:54,993 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 12:58:54,993 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 12:58:54,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.83.20:36203/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 12:58:55,298 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 12:58:55.796090 24908 mini_ranger_kms.cc:310] Created kms service
I20260806 12:58:56.000460 24908 mini_ranger_kms.cc:326] Created kudu user
I20260806 12:58:56.041507 24908 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 12:58:56 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021133, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:58:56.407651 24908 mini_ranger_kms.cc:384] Added ranger policy
I20260806 12:58:56.407768 24908 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-kms:/tmp/dist-test-taskzNDPsb/build/release/bin/postgresql.jar:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskzNDPsb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 12:58:56.407785 24908 mini_ranger_kms.cc:240] Using host: 127.24.83.20
I20260806 12:58:56.411502 24908 mini_ranger_kms.cc:276] Ranger KMS PID: 4898
I20260806 12:58:56.411590 24908 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.83.20:46441
Aug 06, 2026 12:58:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:58:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:58:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 12:58:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 12:58:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 12:58:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 12:58:58 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786021138, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.83.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:58:58.747327 24908 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.368s user 0.009s sys 0.152s
I20260806 12:58:58.747437 24908 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 12:58:58.747483 24908 mini_ranger_kms.cc:401] 127.24.83.20:46441/kms/v1/keys
Aug 06 12:58:59 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021138, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.83.20@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
Aug 06 12:58:59 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021138, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.83.20@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
Aug 06 12:59:01 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:59:01.767057 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:35607
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:35607
--ranger_config_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:46441/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:01.847617 4951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:01.847817 4951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:01.847838 4951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:01.848996 4951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:59:01.849038 4951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:01.849052 4951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:01.849066 4951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:59:01.849077 4951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:59:01.850535 4951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.83.20:46441/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:35607
--ranger_config_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:35607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:01.850709 4951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:01.850957 4951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:59:01.855878 4951 server_base.cc:1034] running on GCE node
W20260806 12:59:01.855861 4959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:01.855949 4956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:01.855863 4957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:01.856390 4951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:01.856685 4951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:01.857849 4951 hybrid_clock.cc:648] HybridClock initialized: now 1786021141857833 us; error 48 us; skew 500 ppm
Aug 06 12:59:01 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021141, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:01.860278 4951 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:59:01.860700 4951 webserver.cc:460] Webserver started at http://127.24.83.62:41139/ using document root <none> and password file <none>
I20260806 12:59:01.860934 4951 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:01.860993 4951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:01.861148 4951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:59:01 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021141, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:59:01.907217 4951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "4743bba30601434aa8482e7ae5249981"
format_stamp: "Formatted at 2026-08-06 12:59:01 on dist-test-slave-zkpd"
server_key: "2a2b6656b437412c0c191ccc55ddb413"
server_key_iv: "0813f8898973b3f7655e06c50a57a0c0"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:01.907680 4951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "4743bba30601434aa8482e7ae5249981"
format_stamp: "Formatted at 2026-08-06 12:59:01 on dist-test-slave-zkpd"
server_key: "2a2b6656b437412c0c191ccc55ddb413"
server_key_iv: "0813f8898973b3f7655e06c50a57a0c0"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:01.909516 4951 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 12:59:01.920578 4966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:01.920750 4951 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:01.920863 4951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "4743bba30601434aa8482e7ae5249981"
format_stamp: "Formatted at 2026-08-06 12:59:01 on dist-test-slave-zkpd"
server_key: "2a2b6656b437412c0c191ccc55ddb413"
server_key_iv: "0813f8898973b3f7655e06c50a57a0c0"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:01.920969 4951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:01.928033 4951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:01.930051 4951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:01.930210 4951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:01.934122 4951 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:35607
I20260806 12:59:01.934213 5020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:35607 every 8 connection(s)
I20260806 12:59:01.934568 4951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:59:01.935325 5021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:01.938436 5021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981: Bootstrap starting.
I20260806 12:59:01.939056 5021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:01.939289 5021 log.cc:824] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:01.939980 5021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981: No bootstrap required, opened a new log
I20260806 12:59:01.941944 5021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } }
I20260806 12:59:01.942050 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 4951
I20260806 12:59:01.942112 5021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:01.942155 5021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4743bba30601434aa8482e7ae5249981, State: Initialized, Role: FOLLOWER
I20260806 12:59:01.942186 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:59:01.942320 5021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } }
I20260806 12:59:01.942365 24908 external_mini_cluster.cc:1428] Setting key 00014c7c9e1d6b06263336e67ff79e39
I20260806 12:59:01.942404 5021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:59:01.942440 5021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:59:01.942493 5021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:01.943168 5021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } }
I20260806 12:59:01.943290 5021 leader_election.cc:302] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4743bba30601434aa8482e7ae5249981; no voters:
I20260806 12:59:01.943464 5021 leader_election.cc:288] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:59:01.943495 5026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:59:01.943639 5026 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 1 LEADER]: Becoming Leader. State: Replica: 4743bba30601434aa8482e7ae5249981, State: Running, Role: LEADER
I20260806 12:59:01.943760 5021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:59:01.943768 5026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } }
I20260806 12:59:01.944232 5027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4743bba30601434aa8482e7ae5249981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } } }
I20260806 12:59:01.944253 5028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4743bba30601434aa8482e7ae5249981. Latest consensus state: current_term: 1 leader_uuid: "4743bba30601434aa8482e7ae5249981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } } }
I20260806 12:59:01.944325 5027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:01.944340 5028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:01.944604 5033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:59:01.945828 5033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:59:01.946162 5021 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 06 12:59:01 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:01.952920 5033 catalog_manager.cc:1347] Generated new cluster ID: 0d749fc180cf4c41b5e635281cd11c40
I20260806 12:59:01.952982 5033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:59:01.957655 5025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:01.942838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42874 (local address 127.24.83.62:35607)
0806 12:59:01.943044 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:01.943054 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:01.943084 (+ 30us) server_negotiation.cc:407] Connection header received
0806 12:59:01.943199 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:01.943206 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:01.943258 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:01.943411 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:01.944210 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:01.946048 (+ 1838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:01.948172 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:01.949657 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:01.949811 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:01.952995 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:01.953001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:01.953001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:01.953015 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:01.955272 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:01.955776 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:01.955779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:01.955780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:01.955932 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:01.956164 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:01.956165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:01.956165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:01.956556 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:01.956715 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:01.957249 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 12:59:01.957403 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260806 12:59:01.975126 5033 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:59:01.975843 5033 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:59:01.984302 5033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981: Generated new TSK 0
I20260806 12:59:01.984472 5033 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:59:02.642995 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:46441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35607
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:02.749445 5067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:02.749672 5067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:02.749712 5067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:02.750905 5067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:02.750979 5067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:02.751025 5067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:59:02.752563 5067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.83.20:46441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:02.752836 5067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:02.753065 5067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:02.758189 5072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:02.758175 5073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:02.758421 5067 server_base.cc:1034] running on GCE node
W20260806 12:59:02.758179 5075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:02.758682 5067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:02.758939 5067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:02.760104 5067 hybrid_clock.cc:648] HybridClock initialized: now 1786021142760091 us; error 32 us; skew 500 ppm
Aug 06 12:59:02 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021142, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:02.762960 5067 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:59:02.763475 5067 webserver.cc:460] Webserver started at http://127.24.83.1:40203/ using document root <none> and password file <none>
I20260806 12:59:02.763731 5067 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:02.763790 5067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:02.763939 5067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:59:02 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021142, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:59:02.776551 5067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "13dbf989c84848d6933d9716ee97e8e2"
format_stamp: "Formatted at 2026-08-06 12:59:02 on dist-test-slave-zkpd"
server_key: "ced88af483ef546e94ba6da56dc99f4c"
server_key_iv: "cf111d815ee9600fe588dd7ecb67ea9b"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:02.776968 5067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "13dbf989c84848d6933d9716ee97e8e2"
format_stamp: "Formatted at 2026-08-06 12:59:02 on dist-test-slave-zkpd"
server_key: "ced88af483ef546e94ba6da56dc99f4c"
server_key_iv: "cf111d815ee9600fe588dd7ecb67ea9b"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:02.779202 5067 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:59:02.797688 5084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:02.797940 5067 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.001s
I20260806 12:59:02.798097 5067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "13dbf989c84848d6933d9716ee97e8e2"
format_stamp: "Formatted at 2026-08-06 12:59:02 on dist-test-slave-zkpd"
server_key: "ced88af483ef546e94ba6da56dc99f4c"
server_key_iv: "cf111d815ee9600fe588dd7ecb67ea9b"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:02.798264 5067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:02.810604 5067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:02.812990 5067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:02.813144 5067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:02.813378 5067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:02.813791 5067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:02.813839 5067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:02.813869 5067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:02.813920 5067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:02.822551 5067 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:43843
I20260806 12:59:02.822659 5201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:43843 every 8 connection(s)
I20260806 12:59:02.823016 5067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:59:02.829099 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 5067
I20260806 12:59:02.829232 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:59:02.829443 24908 external_mini_cluster.cc:1428] Setting key e4f2a0dea9c57e44be90478f47e3b566
Aug 06 12:59:02 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021142, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:02.834784 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:02.824104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:46095 (local address 127.24.83.62:35607)
0806 12:59:02.824254 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:02.824256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:02.824699 (+ 443us) server_negotiation.cc:407] Connection header received
0806 12:59:02.825577 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:02.825578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:02.825594 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:02.825644 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:02.826409 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:02.826959 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:02.828008 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:02.828233 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:02.828303 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:02.830946 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:02.830948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:02.830948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:02.830956 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:02.833004 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:02.833536 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:02.833537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:02.833537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:02.833611 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:02.833948 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:02.833949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:02.833949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:02.834089 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:02.834157 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:02.834538 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 12:59:02.834633 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260806 12:59:02.836383 5205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:02.824075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35607 (local address 127.24.83.1:46095)
0806 12:59:02.824343 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:02.824397 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:02.825433 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:02.825861 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:02.825866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:02.826086 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:02.826287 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:02.826292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:02.827065 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:02.827066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:02.827868 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:02.827870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:02.828338 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:02.828339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:02.828463 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:02.829072 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:02.829083 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:02.830845 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:02.833097 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:02.833104 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:02.833106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:02.833447 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:02.833703 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:02.833704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:02.833705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:02.833866 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:02.834189 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:02.834196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:02.834373 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 12:59:02.834491 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 12:59:02.836110 (+ 1619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":173,"threads_started":1}
I20260806 12:59:02.837172 5202 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35607
I20260806 12:59:02.837292 5202 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:02.837476 5202 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
I20260806 12:59:02.838100 4983 ts_manager.cc:194] Registered new tserver with Master: 13dbf989c84848d6933d9716ee97e8e2 (127.24.83.1:43843)
I20260806 12:59:02.838752 4983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:46095
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:59:02.859076 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:46441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35607
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:59:02 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786021142, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 12:59:02.969467 5208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:02.969831 5208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:02.969935 5208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:02.971658 5208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:02.971756 5208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:02.971840 5208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:59:02.973965 5208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.83.20:46441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:02.974233 5208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:02.974531 5208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:02.981038 5216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:02.981415 5215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:02.981694 5218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:02.983044 5208 server_base.cc:1034] running on GCE node
I20260806 12:59:02.983260 5208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:02.983537 5208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:02.984709 5208 hybrid_clock.cc:648] HybridClock initialized: now 1786021142984692 us; error 24 us; skew 500 ppm
Aug 06 12:59:02 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021142, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:02.987233 5208 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:59:02.987737 5208 webserver.cc:460] Webserver started at http://127.24.83.2:45557/ using document root <none> and password file <none>
I20260806 12:59:02.988042 5208 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:02.988116 5208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:02.988252 5208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:59:02 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021142, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:59:03.009224 5208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "784f5695371c4f8e97e79b0e23b8a66f"
format_stamp: "Formatted at 2026-08-06 12:59:03 on dist-test-slave-zkpd"
server_key: "4da367cc8041ede2f0e3c79c57adea86"
server_key_iv: "5309001f473d30cf96b4c4370f392bdc"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:03.009768 5208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "784f5695371c4f8e97e79b0e23b8a66f"
format_stamp: "Formatted at 2026-08-06 12:59:03 on dist-test-slave-zkpd"
server_key: "4da367cc8041ede2f0e3c79c57adea86"
server_key_iv: "5309001f473d30cf96b4c4370f392bdc"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:03.015441 5208 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260806 12:59:03.030023 5225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.030252 5208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 12:59:03.030366 5208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "784f5695371c4f8e97e79b0e23b8a66f"
format_stamp: "Formatted at 2026-08-06 12:59:03 on dist-test-slave-zkpd"
server_key: "4da367cc8041ede2f0e3c79c57adea86"
server_key_iv: "5309001f473d30cf96b4c4370f392bdc"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:03.030459 5208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:03.053555 5208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:03.056077 5208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:03.056226 5208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:03.056492 5208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:03.056900 5208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:03.056949 5208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.056981 5208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:03.057042 5208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.064641 5208 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:39329
I20260806 12:59:03.065081 5208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:59:03.066701 5344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:39329 every 8 connection(s)
I20260806 12:59:03.067814 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 5208
I20260806 12:59:03.067973 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:59:03.068156 24908 external_mini_cluster.cc:1428] Setting key 67894de6aa6bc7c8dac9edb67d87c0ac
Aug 06 12:59:03 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021142, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:03.077708 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:03.066559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:43715 (local address 127.24.83.62:35607)
0806 12:59:03.066608 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:03.066611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:03.067083 (+ 472us) server_negotiation.cc:407] Connection header received
0806 12:59:03.067852 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:03.067854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:03.067867 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:03.067917 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:03.068925 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.069710 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.071243 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.071584 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.071664 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:03.074226 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:03.074228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:03.074228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:03.074236 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:03.076055 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:03.076546 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:03.076547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:03.076548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:03.076612 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:03.076916 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:03.076917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:03.076917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:03.077057 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:03.077130 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:03.077494 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 12:59:03.077596 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 12:59:03.078500 5347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:03.066266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35607 (local address 127.24.83.2:43715)
0806 12:59:03.066934 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:03.066976 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:03.067686 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:03.068138 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:03.068142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:03.068480 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:03.068783 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:03.068790 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.069835 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.069835 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:03.071124 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:03.071127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.071683 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.071684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:03.071833 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:03.072451 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:03.072461 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:03.074105 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:03.076142 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:03.076149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:03.076152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:03.076456 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:03.076689 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:03.076690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:03.076691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:03.076835 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:03.077136 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:03.077142 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:03.077323 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 12:59:03.078081 (+ 758us) client_negotiation.cc:240] Negotiation successful
0806 12:59:03.078249 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":83,"threads_started":1}
I20260806 12:59:03.079190 5345 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35607
I20260806 12:59:03.079312 5345 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:03.079524 5345 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
I20260806 12:59:03.079960 4983 ts_manager.cc:194] Registered new tserver with Master: 784f5695371c4f8e97e79b0e23b8a66f (127.24.83.2:39329)
I20260806 12:59:03.080453 4983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:43715
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:59:03.104346 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:46441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35607
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:03.224375 5350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:03.224607 5350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:03.224643 5350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:03.225836 5350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:03.225903 5350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:03.225946 5350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:59:03.227448 5350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.83.20:46441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:35607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:03.227720 5350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:03.227977 5350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:03.232918 5356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:03.232928 5355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:03.233144 5350 server_base.cc:1034] running on GCE node
W20260806 12:59:03.233093 5358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:03.233433 5350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:03.233709 5350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:03.234879 5350 hybrid_clock.cc:648] HybridClock initialized: now 1786021143234861 us; error 43 us; skew 500 ppm
Aug 06 12:59:03 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021143, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:03.237521 5350 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:59:03.238015 5350 webserver.cc:460] Webserver started at http://127.24.83.3:35629/ using document root <none> and password file <none>
I20260806 12:59:03.238253 5350 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:03.238304 5350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:03.238447 5350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:59:03 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021143, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:59:03.249774 5350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "efd8223a31d74d58a3de84d794abd59a"
format_stamp: "Formatted at 2026-08-06 12:59:03 on dist-test-slave-zkpd"
server_key: "e3b3b7eb5df91718026263c375c7cf9c"
server_key_iv: "7ea2bb612eec87c7a70c3cdc84e8dd21"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:03.250340 5350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "efd8223a31d74d58a3de84d794abd59a"
format_stamp: "Formatted at 2026-08-06 12:59:03 on dist-test-slave-zkpd"
server_key: "e3b3b7eb5df91718026263c375c7cf9c"
server_key_iv: "7ea2bb612eec87c7a70c3cdc84e8dd21"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:03.252372 5350 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:59:03.272104 5365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.272359 5350 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:03.272454 5350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "efd8223a31d74d58a3de84d794abd59a"
format_stamp: "Formatted at 2026-08-06 12:59:03 on dist-test-slave-zkpd"
server_key: "e3b3b7eb5df91718026263c375c7cf9c"
server_key_iv: "7ea2bb612eec87c7a70c3cdc84e8dd21"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:03.272536 5350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:03.301517 5350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:03.304252 5350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:03.304505 5350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:03.304803 5350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:03.305235 5350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:03.305343 5350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.305428 5350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:03.305488 5350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.322461 5350 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:46461
I20260806 12:59:03.322535 5481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:46461 every 8 connection(s)
I20260806 12:59:03.323007 5350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:59:03.327569 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 5350
I20260806 12:59:03.327685 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:59:03.327914 24908 external_mini_cluster.cc:1428] Setting key c9999dc177d33d32284849e95fede5b6
Aug 06 12:59:03 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021143, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:03.336002 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:03.323842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:60367 (local address 127.24.83.62:35607)
0806 12:59:03.323945 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:03.323948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:03.324247 (+ 299us) server_negotiation.cc:407] Connection header received
0806 12:59:03.324958 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:03.324959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:03.324973 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:03.325005 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:03.325916 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.326751 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.329461 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.329787 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.329864 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:03.332530 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:03.332531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:03.332532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:03.332539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:03.334255 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:03.334759 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:03.334761 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:03.334761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:03.334830 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:03.335152 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:03.335153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:03.335153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:03.335295 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:03.335362 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:03.335782 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 12:59:03.335869 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:59:03.336153 5484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:03.323812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35607 (local address 127.24.83.3:60367)
0806 12:59:03.324091 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:03.324149 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:03.324802 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:03.325210 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:03.325215 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:03.325485 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:03.325796 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:03.325802 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.326870 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.326871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:03.329335 (+ 2464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:03.329338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.329946 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.329947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:03.330092 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:03.330743 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:03.330753 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:03.332430 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:03.334361 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:03.334368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:03.334370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:03.334650 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:03.334922 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:03.334923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:03.334923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:03.335057 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:03.335411 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:03.335418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:03.335607 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 12:59:03.335721 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 12:59:03.335878 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":110,"threads_started":1}
I20260806 12:59:03.336719 5482 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35607
I20260806 12:59:03.336834 5482 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:03.337024 5482 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
I20260806 12:59:03.337365 4983 ts_manager.cc:194] Registered new tserver with Master: efd8223a31d74d58a3de84d794abd59a (127.24.83.3:46461)
I20260806 12:59:03.337833 4983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:60367
I20260806 12:59:03.339692 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:59:03.340445 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 5067
I20260806 12:59:03.351955 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 5208
I20260806 12:59:03.359424 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 5350
I20260806 12:59:03.364743 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 4951
I20260806 12:59:03.370115 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:35607
--webserver_interface=127.24.83.62
--webserver_port=41139
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:35607
--ranger_config_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:46441/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.83.20:46441/kms with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:03.452026 5485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:03.452230 5485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:03.452252 5485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:03.453401 5485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:59:03.453442 5485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:03.453456 5485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:03.453469 5485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:59:03.453518 5485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:59:03.455072 5485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.83.20:46441/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:35607
--ranger_config_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:35607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=41139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:03.455307 5485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:03.455542 5485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:03.460891 5491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:03.460897 5490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:03.460997 5494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:03.461436 5485 server_base.cc:1034] running on GCE node
I20260806 12:59:03.461722 5485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:03.462105 5485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:03.463323 5485 hybrid_clock.cc:648] HybridClock initialized: now 1786021143463291 us; error 41 us; skew 500 ppm
Aug 06 12:59:03 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021143, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:03.466581 5485 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:59:03.467092 5485 webserver.cc:460] Webserver started at http://127.24.83.62:41139/ using document root <none> and password file <none>
I20260806 12:59:03.467337 5485 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:03.467408 5485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 12:59:03 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021143, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:59:03.555181 5485 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:03.556221 5502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.556456 5485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:03.556555 5485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "4743bba30601434aa8482e7ae5249981"
format_stamp: "Formatted at 2026-08-06 12:59:01 on dist-test-slave-zkpd"
server_key: "2a2b6656b437412c0c191ccc55ddb413"
server_key_iv: "0813f8898973b3f7655e06c50a57a0c0"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:03.556949 5485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:03.585237 5485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:03.587044 5485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:03.587229 5485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:03.591738 5485 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:35607
I20260806 12:59:03.591790 5554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:35607 every 8 connection(s)
I20260806 12:59:03.592208 5485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:59:03.593412 5555 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:59:03.594113 5555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981: Bootstrap starting.
I20260806 12:59:03.596323 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 5485
I20260806 12:59:03.596635 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:46441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:43843
--local_ip_for_outbound_sockets=127.24.83.1
--tserver_master_addrs=127.24.83.62:35607
--webserver_port=40203
--webserver_interface=127.24.83.1
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.83.20:46441/kms with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:03.602402 5555 log.cc:824] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:03.604251 5555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:59:03.604552 5555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981: Bootstrap complete.
I20260806 12:59:03.607002 5555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } }
I20260806 12:59:03.607290 5555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4743bba30601434aa8482e7ae5249981, State: Initialized, Role: FOLLOWER
I20260806 12:59:03.607470 5555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } }
I20260806 12:59:03.607544 5555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:59:03.607596 5555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:59:03.607646 5555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:59:03.608803 5555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } }
I20260806 12:59:03.608918 5555 leader_election.cc:302] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4743bba30601434aa8482e7ae5249981; no voters:
I20260806 12:59:03.609114 5555 leader_election.cc:288] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 12:59:03.609149 5559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 12:59:03.609265 5559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [term 2 LEADER]: Becoming Leader. State: Replica: 4743bba30601434aa8482e7ae5249981, State: Running, Role: LEADER
I20260806 12:59:03.609355 5559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } }
I20260806 12:59:03.609377 5555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:59:03.609630 5559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4743bba30601434aa8482e7ae5249981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } } }
I20260806 12:59:03.609701 5559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:03.609813 5559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4743bba30601434aa8482e7ae5249981. Latest consensus state: current_term: 2 leader_uuid: "4743bba30601434aa8482e7ae5249981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4743bba30601434aa8482e7ae5249981" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35607 } } }
I20260806 12:59:03.609891 5559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:03.610311 5563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:59:03.610836 5563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:59:03.610981 5563 catalog_manager.cc:1259] Loaded cluster ID: 0d749fc180cf4c41b5e635281cd11c40
I20260806 12:59:03.611021 5563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:59:03.611299 5555 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 12:59:03.614905 5563 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:59:03.615530 5563 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981: Loaded TSK: 0
I20260806 12:59:03.615913 5563 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 12:59:03.706168 5557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:03.706436 5557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:03.706477 5557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:03.708145 5557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:03.708201 5557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:03.708252 5557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:59:03.710412 5557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.83.20:46441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:43843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=40203
--tserver_master_addrs=127.24.83.62:35607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:03.710660 5557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:03.710937 5557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:03.717217 5593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:03.717202 5596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:03.717202 5594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:03.718057 5557 server_base.cc:1034] running on GCE node
I20260806 12:59:03.718278 5557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:03.718566 5557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:03.721884 5557 hybrid_clock.cc:648] HybridClock initialized: now 1786021143721855 us; error 38 us; skew 500 ppm
Aug 06 12:59:03 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021143, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:03.728627 5557 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:59:03.729080 5557 webserver.cc:460] Webserver started at http://127.24.83.1:40203/ using document root <none> and password file <none>
I20260806 12:59:03.729346 5557 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:03.729426 5557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 12:59:03 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021143, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:59:03.758241 5557 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:03.758957 5604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.759150 5557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:03.759246 5557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "13dbf989c84848d6933d9716ee97e8e2"
format_stamp: "Formatted at 2026-08-06 12:59:02 on dist-test-slave-zkpd"
server_key: "ced88af483ef546e94ba6da56dc99f4c"
server_key_iv: "cf111d815ee9600fe588dd7ecb67ea9b"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:03.759706 5557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:03.790663 5557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:03.793181 5557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:03.793408 5557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:03.793744 5557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:03.794276 5557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:03.794368 5557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.794469 5557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:03.794544 5557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.805377 5557 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:43843
I20260806 12:59:03.805913 5557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:59:03.806803 5721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:43843 every 8 connection(s)
I20260806 12:59:03.814778 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 5557
I20260806 12:59:03.815100 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:46441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:39329
--local_ip_for_outbound_sockets=127.24.83.2
--tserver_master_addrs=127.24.83.62:35607
--webserver_port=45557
--webserver_interface=127.24.83.2
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.83.20:46441/kms with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:59:03 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021143, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:03.821786 5725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:03.807753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:40051 (local address 127.24.83.62:35607)
0806 12:59:03.808020 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:03.808029 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:03.808067 (+ 38us) server_negotiation.cc:407] Connection header received
0806 12:59:03.808913 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:03.808921 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:03.808981 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:03.809117 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:03.810071 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.811465 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.813031 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.813667 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.813825 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:03.817384 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:03.817389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:03.817389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:03.817403 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:03.819648 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:03.820173 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:03.820177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:03.820177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:03.820281 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:03.820550 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:03.820551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:03.820552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:03.820779 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:03.820873 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:03.821453 (+ 580us) server_negotiation.cc:299] Negotiation successful
0806 12:59:03.821573 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":99,"threads_started":1}
I20260806 12:59:03.822271 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:03.807654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35607 (local address 127.24.83.1:40051)
0806 12:59:03.807867 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:03.807926 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:03.808609 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:03.809304 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:03.809308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:03.809589 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:03.809914 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:03.809941 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.811609 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.811610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:03.812919 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:03.812922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:03.813817 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:03.813818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:03.813975 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:03.814622 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:03.814633 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:03.817254 (+ 2621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:03.819743 (+ 2489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:03.819749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:03.819751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:03.820071 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:03.820368 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:03.820369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:03.820369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:03.820476 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:03.820853 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:03.820859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:03.821004 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 12:59:03.821105 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 12:59:03.821257 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260806 12:59:03.823439 5722 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35607
I20260806 12:59:03.823536 5722 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:03.823734 5722 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
W20260806 12:59:03.824731 5722 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:35607 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 12:59:03.824817 5722 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:03.824885 5722 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
I20260806 12:59:03.825274 5722 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:03.825352 5722 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
W20260806 12:59:03.825681 5722 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:59:03.938459 5726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:03.938807 5726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:03.938882 5726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:03.940613 5726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:03.940737 5726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:03.940802 5726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:59:03.943089 5726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.83.20:46441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:39329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=45557
--tserver_master_addrs=127.24.83.62:35607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:03.943331 5726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:03.943598 5726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:03.949151 5732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:03.949678 5731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:03.949908 5726 server_base.cc:1034] running on GCE node
W20260806 12:59:03.949970 5734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:03.950258 5726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:03.950525 5726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:03.951705 5726 hybrid_clock.cc:648] HybridClock initialized: now 1786021143951696 us; error 26 us; skew 500 ppm
Aug 06 12:59:03 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021143, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:03.954294 5726 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:59:03.954777 5726 webserver.cc:460] Webserver started at http://127.24.83.2:45557/ using document root <none> and password file <none>
I20260806 12:59:03.955021 5726 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:03.955099 5726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 12:59:03 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021143, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:59:03.970443 5726 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:03.971189 5742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.971414 5726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 12:59:03.971529 5726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "784f5695371c4f8e97e79b0e23b8a66f"
format_stamp: "Formatted at 2026-08-06 12:59:03 on dist-test-slave-zkpd"
server_key: "4da367cc8041ede2f0e3c79c57adea86"
server_key_iv: "5309001f473d30cf96b4c4370f392bdc"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:03.972057 5726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:03.989542 5726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:03.991708 5726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:03.991878 5726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:03.992148 5726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:03.992456 5726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:03.992539 5726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.992578 5726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:03.992606 5726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:03.998612 5726 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:39329
I20260806 12:59:03.998740 5858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:39329 every 8 connection(s)
I20260806 12:59:03.999094 5726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:59:04 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021143, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:04.006469 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 5726
I20260806 12:59:04.006794 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.83.20:46441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:46461
--local_ip_for_outbound_sockets=127.24.83.3
--tserver_master_addrs=127.24.83.62:35607
--webserver_port=35629
--webserver_interface=127.24.83.3
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.83.20:46441/kms with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:04.011651 5725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:04.000050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:55635 (local address 127.24.83.62:35607)
0806 12:59:04.000126 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:04.000128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:04.000389 (+ 261us) server_negotiation.cc:407] Connection header received
0806 12:59:04.001004 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:04.001006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:04.001023 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:04.001090 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:04.001900 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:04.002394 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:04.003410 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:04.003658 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:04.003692 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:04.008128 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:04.008130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:04.008130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:04.008138 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:04.009966 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:04.010476 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:04.010478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:04.010478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:04.010542 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:04.010808 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:04.010809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:04.010810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:04.010956 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:04.011028 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:04.011413 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 12:59:04.011511 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:59:04.011799 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:03.999867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35607 (local address 127.24.83.2:55635)
0806 12:59:04.000222 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:04.000282 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:04.000867 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:04.001228 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:04.001232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:04.001479 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:04.001761 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:04.001768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:04.002494 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:04.002494 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:04.003301 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:04.003304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:04.003772 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:04.003773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:04.003862 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:04.004352 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:04.004377 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:04.008022 (+ 3645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:04.010052 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:04.010070 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:04.010072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:04.010386 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:04.010608 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:04.010609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:04.010609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:04.010740 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:04.011058 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:04.011065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:04.011247 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 12:59:04.011365 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 12:59:04.011531 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":114,"threads_started":1}
I20260806 12:59:04.012312 5859 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35607
I20260806 12:59:04.012414 5859 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:04.012621 5859 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
W20260806 12:59:04.013267 5859 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:35607 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 12:59:04.013368 5859 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:04.013499 5859 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
I20260806 12:59:04.013854 5859 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:04.013976 5859 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
W20260806 12:59:04.014292 5859 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:59:04.106125 5862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:04.106457 5862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:04.106546 5862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:04.108556 5862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:04.108675 5862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:04.108774 5862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:59:04.111333 5862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.83.20:46441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:46461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=35629
--tserver_master_addrs=127.24.83.62:35607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:04.111872 5862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:04.112257 5862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:04.119366 5871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:04.119459 5868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:04.119369 5869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:04.121116 5862 server_base.cc:1034] running on GCE node
I20260806 12:59:04.121366 5862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:04.121661 5862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:04.122820 5862 hybrid_clock.cc:648] HybridClock initialized: now 1786021144122814 us; error 48 us; skew 500 ppm
Aug 06 12:59:04 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:04.125433 5862 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:59:04.125964 5862 webserver.cc:460] Webserver started at http://127.24.83.3:35629/ using document root <none> and password file <none>
I20260806 12:59:04.126273 5862 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:04.126369 5862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 12:59:04 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:59:04.153440 5862 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:04.154306 5879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:04.154511 5862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:04.154603 5862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "efd8223a31d74d58a3de84d794abd59a"
format_stamp: "Formatted at 2026-08-06 12:59:03 on dist-test-slave-zkpd"
server_key: "e3b3b7eb5df91718026263c375c7cf9c"
server_key_iv: "7ea2bb612eec87c7a70c3cdc84e8dd21"
server_key_version: "kuduclusterkey@0"
I20260806 12:59:04.155067 5862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:04.178973 5862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:04.181574 5862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:04.181743 5862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:04.182019 5862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:04.182420 5862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:04.182474 5862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:04.182507 5862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:04.182535 5862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:04.189888 5862 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:46461
I20260806 12:59:04.189991 5995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:46461 every 8 connection(s)
I20260806 12:59:04.190375 5862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:59:04.195163 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 5862
Aug 06 12:59:04 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:04.201241 5998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:04.191224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:35607 (local address 127.24.83.3:41891)
0806 12:59:04.191427 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:04.191475 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:04.192016 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:04.192290 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:04.192294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:04.192467 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:04.192656 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:04.192660 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:04.193642 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:04.193643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:04.194608 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:04.194611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:04.195120 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:04.195121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:04.195277 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:04.195830 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:04.195841 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:04.197612 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:04.199549 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:04.199556 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:04.199558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:04.199853 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:04.200102 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:04.200102 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:04.200103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:04.200233 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:04.200570 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:04.200577 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:04.200762 (+ 185us) client_negotiation.cc:769] Sending connection context
0806 12:59:04.200866 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 12:59:04.201014 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
I20260806 12:59:04.201866 5725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:04.191233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:41891 (local address 127.24.83.62:35607)
0806 12:59:04.191285 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:04.191287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:04.191587 (+ 300us) server_negotiation.cc:407] Connection header received
0806 12:59:04.192126 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:04.192128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:04.192141 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:04.192218 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:04.192759 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:04.193541 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:04.194709 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:04.195009 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:04.195088 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:04.197732 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:04.197734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:04.197734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:04.197754 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:04.199454 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:04.199952 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:04.199953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:04.199954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:04.200019 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:04.200332 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:04.200334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:04.200334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:04.200480 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:04.200564 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:04.201737 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0806 12:59:04.201807 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 12:59:04.202376 5996 heartbeater.cc:347] Connected to a master server at 127.24.83.62:35607
I20260806 12:59:04.202497 5996 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:04.202696 5996 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
W20260806 12:59:04.203450 5996 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:35607 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 12:59:04.203531 5996 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:04.203614 5996 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
I20260806 12:59:04.204044 5996 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:04.204133 5996 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
W20260806 12:59:04.204550 5996 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 12:59:04.207523 6000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:04.195868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.24.83.62:35607)
0806 12:59:04.196591 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:04.196593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:04.196601 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:59:04.196623 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:04.196624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:04.196637 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:04.196710 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:04.199651 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:04.200359 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:04.201971 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:04.202261 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:04.202330 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:04.204929 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:04.204930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:04.204930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:04.204936 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:04.206364 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:04.206717 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:04.206718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:04.206718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:04.206790 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:04.206993 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:04.206994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:04.206995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:04.207097 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:04.207293 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:04.207331 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 12:59:04.207406 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":91,"threads_started":1}
Aug 06 12:59:04 dist-test-slave-zkpd krb5kdc[4501](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786021144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:04.825909 5722 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:04.826062 5722 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
I20260806 12:59:04.826475 5519 ts_manager.cc:194] Registered new tserver with Master: 13dbf989c84848d6933d9716ee97e8e2 (127.24.83.1:43843)
I20260806 12:59:04.827482 5519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:40051
I20260806 12:59:05.014492 5859 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:05.014628 5859 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
I20260806 12:59:05.015002 5519 ts_manager.cc:194] Registered new tserver with Master: 784f5695371c4f8e97e79b0e23b8a66f (127.24.83.2:39329)
I20260806 12:59:05.015501 5519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:55635
Aug 06 12:59:05 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:59:05.204721 5996 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:05.204877 5996 heartbeater.cc:511] Master 127.24.83.62:35607 requested a full tablet report, sending...
I20260806 12:59:05.205235 5519 ts_manager.cc:194] Registered new tserver with Master: efd8223a31d74d58a3de84d794abd59a (127.24.83.3:46461)
I20260806 12:59:05.205742 5519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:41891
I20260806 12:59:05.212991 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:59:05.222431 6017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.214225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42884 (local address 127.24.83.62:35607)
0806 12:59:05.214506 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:05.214509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:05.214661 (+ 152us) server_negotiation.cc:407] Connection header received
0806 12:59:05.214684 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:05.214685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:05.214696 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:05.214751 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:05.215448 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.216310 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.217361 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.217667 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.217729 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.219205 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:05.219207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:05.219208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:05.219215 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:05.220981 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:05.221358 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:05.221359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:05.221360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:05.221415 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:05.221592 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:05.221593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:05.221594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:05.221709 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:05.221805 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:05.222212 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 12:59:05.222290 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":59,"threads_started":1}
I20260806 12:59:05.224712 5519 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:59:05.226863 5519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:59:05.245869 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.238467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44782 (local address 127.24.83.3:46461)
0806 12:59:05.239081 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:05.239086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:05.239319 (+ 233us) server_negotiation.cc:407] Connection header received
0806 12:59:05.239417 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:05.239419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:05.239436 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:05.239488 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:05.239729 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.241177 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.243973 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.244704 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.244780 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.244860 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:05.245731 (+ 871us) server_negotiation.cc:299] Negotiation successful
0806 12:59:05.245776 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":45,"threads_started":1}
I20260806 12:59:05.246343 6024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.238247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:46461 (local address 127.0.0.1:44782)
0806 12:59:05.239269 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:05.239277 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:05.239341 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:05.239589 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:05.239592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:05.239594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:05.239656 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.239658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.241269 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.241269 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.242563 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.242565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.244926 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.244927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.245010 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.245024 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 12:59:05.246200 (+ 1176us) client_negotiation.cc:240] Negotiation successful
0806 12:59:05.246266 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1011,"thread_start_us":69,"threads_started":1}
I20260806 12:59:05.247211 5927 tablet_service.cc:1467] Processing CreateTablet for tablet cb8f6da8405c40c289134e49191bf2f2 (DEFAULT_TABLE table=test-table [id=762ddeff2f3f456ea8cea54dcdfb064b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:05.247604 5927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb8f6da8405c40c289134e49191bf2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:05.249581 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.237871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:43843 (local address 127.0.0.1:37180)
0806 12:59:05.238856 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:05.238898 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:05.238958 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:05.243138 (+ 4180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:05.243139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:05.243141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:05.243180 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.243181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.247105 (+ 3924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.247106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.248144 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.248147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.249367 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.249368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.249429 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.249433 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:05.249464 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 12:59:05.249502 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":956,"thread_start_us":73,"threads_started":1}
I20260806 12:59:05.249994 6025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.237974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37180 (local address 127.24.83.1:43843)
0806 12:59:05.242973 (+ 4999us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:05.242978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:05.242989 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:59:05.243023 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:05.243026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:05.243041 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:05.243242 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:05.243315 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.247009 (+ 3694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.248251 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.249268 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.249753 (+ 485us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.249851 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:05.249878 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 12:59:05.249920 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4690,"thread_start_us":68,"threads_started":1}
I20260806 12:59:05.250655 5654 tablet_service.cc:1467] Processing CreateTablet for tablet cb8f6da8405c40c289134e49191bf2f2 (DEFAULT_TABLE table=test-table [id=762ddeff2f3f456ea8cea54dcdfb064b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:05.250984 5654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb8f6da8405c40c289134e49191bf2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:05.251700 6026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.238101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:39329 (local address 127.0.0.1:39740)
0806 12:59:05.239124 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:05.239132 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:05.239167 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:05.248184 (+ 9017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:05.248185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:05.248187 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:05.248235 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.248237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.249509 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.249510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.250551 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.250554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.251491 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.251492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.251556 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.251560 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:05.251592 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 12:59:05.251627 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"thread_start_us":70,"threads_started":1}
I20260806 12:59:05.252121 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.238197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39740 (local address 127.24.83.2:39329)
0806 12:59:05.247985 (+ 9788us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:05.247990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:05.248002 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:59:05.248039 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:05.248041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:05.248058 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:05.248306 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:05.248383 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.249404 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.250660 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.251376 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.251864 (+ 488us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.251974 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:05.252003 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 12:59:05.252049 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9749,"thread_start_us":42,"threads_started":1}
I20260806 12:59:05.252684 5791 tablet_service.cc:1467] Processing CreateTablet for tablet cb8f6da8405c40c289134e49191bf2f2 (DEFAULT_TABLE table=test-table [id=762ddeff2f3f456ea8cea54dcdfb064b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:05.253042 5791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb8f6da8405c40c289134e49191bf2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:05.264041 6032 tablet_bootstrap.cc:492] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f: Bootstrap starting.
I20260806 12:59:05.264793 6032 tablet_bootstrap.cc:654] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:05.265143 6032 log.cc:824] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:05.265511 6031 tablet_bootstrap.cc:492] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a: Bootstrap starting.
I20260806 12:59:05.266161 6031 tablet_bootstrap.cc:654] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:05.266439 6031 log.cc:824] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:05.268016 6030 tablet_bootstrap.cc:492] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2: Bootstrap starting.
I20260806 12:59:05.268616 6030 tablet_bootstrap.cc:654] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:05.268895 6030 log.cc:824] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:05.269737 6032 tablet_bootstrap.cc:492] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f: No bootstrap required, opened a new log
I20260806 12:59:05.269853 6032 ts_tablet_manager.cc:1397] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 12:59:05.270960 6030 tablet_bootstrap.cc:492] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2: No bootstrap required, opened a new log
I20260806 12:59:05.271085 6030 ts_tablet_manager.cc:1397] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 12:59:05.272172 6032 raft_consensus.cc:348] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 } } peers { permanent_uuid: "784f5695371c4f8e97e79b0e23b8a66f" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39329 } } peers { permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 } }
I20260806 12:59:05.272753 6032 raft_consensus.cc:374] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:05.272842 6032 raft_consensus.cc:729] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784f5695371c4f8e97e79b0e23b8a66f, State: Initialized, Role: FOLLOWER
I20260806 12:59:05.273041 6032 consensus_queue.cc:260] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 } } peers { permanent_uuid: "784f5695371c4f8e97e79b0e23b8a66f" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39329 } } peers { permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 } }
I20260806 12:59:05.273398 6032 ts_tablet_manager.cc:1428] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 12:59:05.273554 5859 heartbeater.cc:503] Master 127.24.83.62:35607 was elected leader, sending a full tablet report...
I20260806 12:59:05.273555 6030 raft_consensus.cc:348] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 } } peers { permanent_uuid: "784f5695371c4f8e97e79b0e23b8a66f" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39329 } } peers { permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 } }
I20260806 12:59:05.273725 6030 raft_consensus.cc:374] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:05.273754 6030 raft_consensus.cc:729] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13dbf989c84848d6933d9716ee97e8e2, State: Initialized, Role: FOLLOWER
I20260806 12:59:05.273851 6030 consensus_queue.cc:260] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 } } peers { permanent_uuid: "784f5695371c4f8e97e79b0e23b8a66f" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39329 } } peers { permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 } }
I20260806 12:59:05.274072 6030 ts_tablet_manager.cc:1428] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 12:59:05.274175 5722 heartbeater.cc:503] Master 127.24.83.62:35607 was elected leader, sending a full tablet report...
I20260806 12:59:05.275204 6031 tablet_bootstrap.cc:492] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a: No bootstrap required, opened a new log
I20260806 12:59:05.275269 6031 ts_tablet_manager.cc:1397] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260806 12:59:05.278103 6031 raft_consensus.cc:348] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 } } peers { permanent_uuid: "784f5695371c4f8e97e79b0e23b8a66f" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39329 } } peers { permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 } }
I20260806 12:59:05.278335 6031 raft_consensus.cc:374] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:05.278414 6031 raft_consensus.cc:729] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd8223a31d74d58a3de84d794abd59a, State: Initialized, Role: FOLLOWER
I20260806 12:59:05.278587 6031 consensus_queue.cc:260] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 } } peers { permanent_uuid: "784f5695371c4f8e97e79b0e23b8a66f" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39329 } } peers { permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 } }
I20260806 12:59:05.278895 6031 ts_tablet_manager.cc:1428] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 12:59:05.279042 5996 heartbeater.cc:503] Master 127.24.83.62:35607 was elected leader, sending a full tablet report...
I20260806 12:59:05.292903 6037 raft_consensus.cc:482] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:59:05.293035 6037 raft_consensus.cc:504] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 } } peers { permanent_uuid: "784f5695371c4f8e97e79b0e23b8a66f" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39329 } } peers { permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 } }
I20260806 12:59:05.293388 6037 leader_election.cc:288] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 784f5695371c4f8e97e79b0e23b8a66f (127.24.83.2:39329), efd8223a31d74d58a3de84d794abd59a (127.24.83.3:46461)
I20260806 12:59:05.293401 6036 raft_consensus.cc:482] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:59:05.293624 6036 raft_consensus.cc:504] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 } } peers { permanent_uuid: "784f5695371c4f8e97e79b0e23b8a66f" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39329 } } peers { permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 } }
I20260806 12:59:05.296653 6036 leader_election.cc:288] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13dbf989c84848d6933d9716ee97e8e2 (127.24.83.1:43843), efd8223a31d74d58a3de84d794abd59a (127.24.83.3:46461)
I20260806 12:59:05.300591 6025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.294777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:60101 (local address 127.24.83.1:43843)
0806 12:59:05.294815 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:05.294817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:05.296919 (+ 2102us) server_negotiation.cc:407] Connection header received
0806 12:59:05.297022 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:05.297023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:05.297034 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:05.297064 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:05.297319 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.298044 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.299578 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.300186 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.300236 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.300247 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:05.300478 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 12:59:05.300521 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 12:59:05.301033 6042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.294614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:43843 (local address 127.24.83.2:60101)
0806 12:59:05.296848 (+ 2234us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:05.296860 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:05.296948 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:05.297132 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:05.297133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:05.297135 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:05.297229 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.297231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.298153 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.298154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.299385 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.299389 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.300320 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.300320 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.300394 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.300398 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:05.300885 (+ 487us) client_negotiation.cc:240] Negotiation successful
0806 12:59:05.300932 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2217,"thread_start_us":79,"threads_started":1}
I20260806 12:59:05.301502 5667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8f6da8405c40c289134e49191bf2f2" candidate_uuid: "784f5695371c4f8e97e79b0e23b8a66f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13dbf989c84848d6933d9716ee97e8e2" is_pre_election: true
I20260806 12:59:05.301620 5667 raft_consensus.cc:2454] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 784f5695371c4f8e97e79b0e23b8a66f in term 0.
I20260806 12:59:05.301931 5745 leader_election.cc:302] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13dbf989c84848d6933d9716ee97e8e2, 784f5695371c4f8e97e79b0e23b8a66f; no voters:
I20260806 12:59:05.302058 6036 raft_consensus.cc:2781] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:59:05.302096 6036 raft_consensus.cc:482] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:59:05.302117 6036 raft_consensus.cc:3037] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:05.303036 6036 raft_consensus.cc:504] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 } } peers { permanent_uuid: "784f5695371c4f8e97e79b0e23b8a66f" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39329 } } peers { permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 } }
I20260806 12:59:05.303165 6036 leader_election.cc:288] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [CANDIDATE]: Term 1 election: Requested vote from peers 13dbf989c84848d6933d9716ee97e8e2 (127.24.83.1:43843), efd8223a31d74d58a3de84d794abd59a (127.24.83.3:46461)
I20260806 12:59:05.303496 5667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8f6da8405c40c289134e49191bf2f2" candidate_uuid: "784f5695371c4f8e97e79b0e23b8a66f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13dbf989c84848d6933d9716ee97e8e2"
I20260806 12:59:05.303567 5667 raft_consensus.cc:3037] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:05.304509 5667 raft_consensus.cc:2454] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784f5695371c4f8e97e79b0e23b8a66f in term 1.
I20260806 12:59:05.304718 5745 leader_election.cc:302] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13dbf989c84848d6933d9716ee97e8e2, 784f5695371c4f8e97e79b0e23b8a66f; no voters:
I20260806 12:59:05.304834 6036 raft_consensus.cc:2781] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:59:05.304975 6036 raft_consensus.cc:686] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 1 LEADER]: Becoming Leader. State: Replica: 784f5695371c4f8e97e79b0e23b8a66f, State: Running, Role: LEADER
I20260806 12:59:05.305056 6036 consensus_queue.cc:237] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 } } peers { permanent_uuid: "784f5695371c4f8e97e79b0e23b8a66f" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39329 } } peers { permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 } }
I20260806 12:59:05.305962 5518 catalog_manager.cc:5495] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f reported cstate change: term changed from 0 to 1, leader changed from <none> to 784f5695371c4f8e97e79b0e23b8a66f (127.24.83.2). New cstate: current_term: 1 leader_uuid: "784f5695371c4f8e97e79b0e23b8a66f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784f5695371c4f8e97e79b0e23b8a66f" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 39329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 } health_report { overall_health: UNKNOWN } } }
W20260806 12:59:05.308012 5723 tablet.cc:2369] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:59:05.311612 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.294025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:52549 (local address 127.24.83.2:39329)
0806 12:59:05.294063 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:05.294066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:05.298842 (+ 4776us) server_negotiation.cc:407] Connection header received
0806 12:59:05.298957 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:05.298957 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:05.298968 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:05.298997 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:05.299242 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.299937 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.309752 (+ 9815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.310268 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.310316 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.310326 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:05.311368 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0806 12:59:05.311413 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 12:59:05.312721 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.293611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:39329 (local address 127.24.83.1:52549)
0806 12:59:05.298766 (+ 5155us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:05.298780 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:05.298870 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:05.299062 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:05.299062 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:05.299064 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:05.299157 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.299159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.300039 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.300040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.301236 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.301240 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.310389 (+ 9149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.310390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.310462 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.310466 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:05.312573 (+ 2107us) client_negotiation.cc:240] Negotiation successful
0806 12:59:05.312625 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5106,"thread_start_us":127,"threads_started":1}
I20260806 12:59:05.313162 5812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8f6da8405c40c289134e49191bf2f2" candidate_uuid: "13dbf989c84848d6933d9716ee97e8e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784f5695371c4f8e97e79b0e23b8a66f" is_pre_election: true
I20260806 12:59:05.313546 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.294047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:47071 (local address 127.24.83.3:46461)
0806 12:59:05.294092 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:05.294094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:05.308233 (+ 14139us) server_negotiation.cc:407] Connection header received
0806 12:59:05.309495 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:05.309497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:05.309508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:05.309536 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:05.310552 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.311221 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.312588 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.313113 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.313163 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.313173 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:05.313439 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 12:59:05.313482 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:59:05.313928 6039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.293839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:46461 (local address 127.24.83.1:47071)
0806 12:59:05.308174 (+ 14335us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:05.308185 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:05.309412 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:05.309593 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:05.309594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:05.309596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:05.309663 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.309664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.311295 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.311296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.312473 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.312477 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.313274 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.313275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.313351 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.313355 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:05.313805 (+ 450us) client_negotiation.cc:240] Negotiation successful
0806 12:59:05.313853 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14315,"thread_start_us":66,"threads_started":1}
I20260806 12:59:05.313946 6041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.296362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:46461 (local address 127.24.83.2:38815)
0806 12:59:05.306795 (+ 10433us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:05.306808 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:05.306874 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:05.307114 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:05.307115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:05.307116 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:05.307184 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.307186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.308268 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.308268 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.309315 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:05.309318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.313648 (+ 4330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.313649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:05.313745 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.313749 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:05.313812 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 12:59:05.313851 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10411,"thread_start_us":76,"threads_started":1}
I20260806 12:59:05.314221 6043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.296555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:38815 (local address 127.24.83.3:46461)
0806 12:59:05.299526 (+ 2971us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:05.299528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:05.306945 (+ 7417us) server_negotiation.cc:407] Connection header received
0806 12:59:05.306978 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:05.306979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:05.306991 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:05.307050 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:05.307268 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.308044 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.311921 (+ 3877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.312471 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.312543 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.312554 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:05.314104 (+ 1550us) server_negotiation.cc:299] Negotiation successful
0806 12:59:05.314148 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2946,"thread_start_us":72,"threads_started":1}
I20260806 12:59:05.314556 5948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8f6da8405c40c289134e49191bf2f2" candidate_uuid: "13dbf989c84848d6933d9716ee97e8e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd8223a31d74d58a3de84d794abd59a" is_pre_election: true
I20260806 12:59:05.314582 5947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8f6da8405c40c289134e49191bf2f2" candidate_uuid: "784f5695371c4f8e97e79b0e23b8a66f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd8223a31d74d58a3de84d794abd59a" is_pre_election: true
I20260806 12:59:05.314708 5948 raft_consensus.cc:2454] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13dbf989c84848d6933d9716ee97e8e2 in term 0.
I20260806 12:59:05.314810 5946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8f6da8405c40c289134e49191bf2f2" candidate_uuid: "784f5695371c4f8e97e79b0e23b8a66f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd8223a31d74d58a3de84d794abd59a"
I20260806 12:59:05.314872 5946 raft_consensus.cc:3037] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:05.314880 5608 leader_election.cc:302] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13dbf989c84848d6933d9716ee97e8e2, efd8223a31d74d58a3de84d794abd59a; no voters: 784f5695371c4f8e97e79b0e23b8a66f
I20260806 12:59:05.315758 5946 raft_consensus.cc:2454] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784f5695371c4f8e97e79b0e23b8a66f in term 1.
I20260806 12:59:05.315850 6037 raft_consensus.cc:2741] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 12:59:05.320024 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.316728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39752 (local address 127.24.83.2:39329)
0806 12:59:05.316764 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:05.316766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:05.316845 (+ 79us) server_negotiation.cc:407] Connection header received
0806 12:59:05.316904 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:05.316905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:05.316924 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:05.316988 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:59:05.317153 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.317816 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.318920 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:05.319172 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:05.319231 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:05.319449 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:59:05.319574 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:59:05.319654 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:05.319933 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 12:59:05.319956 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 12:59:05.324046 5946 raft_consensus.cc:1261] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a [term 1 FOLLOWER]: Refusing update from remote peer 784f5695371c4f8e97e79b0e23b8a66f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:05.324153 5667 raft_consensus.cc:1261] T cb8f6da8405c40c289134e49191bf2f2 P 13dbf989c84848d6933d9716ee97e8e2 [term 1 FOLLOWER]: Refusing update from remote peer 784f5695371c4f8e97e79b0e23b8a66f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:05.324419 6036 consensus_queue.cc:1035] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [LEADER]: Connected to new peer: Peer: permanent_uuid: "13dbf989c84848d6933d9716ee97e8e2" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:05.324596 6036 consensus_queue.cc:1035] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd8223a31d74d58a3de84d794abd59a" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 46461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:05.329099 6047 mvcc.cc:205] Tried to move back new op lower bound from 7315542611244032000 to 7315542611169951744. Current Snapshot: MvccSnapshot[applied={T|T < 7315542611244032000}]
I20260806 12:59:05.329428 6050 mvcc.cc:205] Tried to move back new op lower bound from 7315542611244032000 to 7315542611169951744. Current Snapshot: MvccSnapshot[applied={T|T < 7315542611244032000}]
I20260806 12:59:05.331259 6049 mvcc.cc:205] Tried to move back new op lower bound from 7315542611244032000 to 7315542611169951744. Current Snapshot: MvccSnapshot[applied={T|T < 7315542611244032000}]
I20260806 12:59:05.332356 5519 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:59:05.332449 5519 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:59:05.333218 5519 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4743bba30601434aa8482e7ae5249981: Sending DeleteTablet for 3 replicas of tablet cb8f6da8405c40c289134e49191bf2f2
I20260806 12:59:05.333563 5654 tablet_service.cc:1514] Processing DeleteTablet for tablet cb8f6da8405c40c289134e49191bf2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:05 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:37180
I20260806 12:59:05.334018 5927 tablet_service.cc:1514] Processing DeleteTablet for tablet cb8f6da8405c40c289134e49191bf2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:05 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:44782
I20260806 12:59:05.334421 5791 tablet_service.cc:1514] Processing DeleteTablet for tablet cb8f6da8405c40c289134e49191bf2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:05 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:39740
I20260806 12:59:05.334903 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 5557
I20260806 12:59:05.339461 6060 tablet_replica.cc:330] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f: stopping tablet replica
I20260806 12:59:05.339600 6059 tablet_replica.cc:330] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a: stopping tablet replica
I20260806 12:59:05.339756 6059 raft_consensus.cc:2229] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:59:05.339881 6060 raft_consensus.cc:2229] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:59:05.340061 6059 raft_consensus.cc:2258] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:05.340181 6060 raft_consensus.cc:2258] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:05.340741 6059 ts_tablet_manager.cc:1905] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:59:05.341974 6060 ts_tablet_manager.cc:1905] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:59:05.352567 6060 ts_tablet_manager.cc:1918] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:05.352658 6060 log.cc:1196] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/cb8f6da8405c40c289134e49191bf2f2
I20260806 12:59:05.352949 6060 ts_tablet_manager.cc:1939] T cb8f6da8405c40c289134e49191bf2f2 P 784f5695371c4f8e97e79b0e23b8a66f: Deleting consensus metadata
I20260806 12:59:05.353449 5506 catalog_manager.cc:4841] TS 784f5695371c4f8e97e79b0e23b8a66f (127.24.83.2:39329): tablet cb8f6da8405c40c289134e49191bf2f2 (table test-table [id=762ddeff2f3f456ea8cea54dcdfb064b]) successfully deleted
I20260806 12:59:05.353664 6059 ts_tablet_manager.cc:1918] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:05.353770 6059 log.cc:1196] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/cb8f6da8405c40c289134e49191bf2f2
I20260806 12:59:05.354048 6059 ts_tablet_manager.cc:1939] T cb8f6da8405c40c289134e49191bf2f2 P efd8223a31d74d58a3de84d794abd59a: Deleting consensus metadata
I20260806 12:59:05.354475 5506 catalog_manager.cc:4841] TS efd8223a31d74d58a3de84d794abd59a (127.24.83.3:46461): tablet cb8f6da8405c40c289134e49191bf2f2 (table test-table [id=762ddeff2f3f456ea8cea54dcdfb064b]) successfully deleted
W20260806 12:59:05.358116 5505 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:43843 (error 108)
I20260806 12:59:05.359203 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 5726
Aug 06 12:59:05 dist-test-slave-zkpd krb5kdc[4501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for HTTP/127.24.83.20@KRBTEST.COM
I20260806 12:59:05.366896 6026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:05.366728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:43843 (local address 127.0.0.1:37196)
0806 12:59:05.366786 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:05.366842 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 12:59:05.368083 5505 catalog_manager.cc:4584] TS 13dbf989c84848d6933d9716ee97e8e2 (127.24.83.1:43843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb8f6da8405c40c289134e49191bf2f2: Network error: Client connection negotiation failed: client connection to 127.24.83.1:43843: connect: Connection refused (error 111)
I20260806 12:59:05.379283 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 5862
I20260806 12:59:05.389151 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 5485
I20260806 12:59:05.398001 24908 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 12:59:05.752621 24908 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 12:59:05.752727 24908 mini_ranger.cc:66] Stopping Ranger...
I20260806 12:59:06.097345 24908 mini_ranger.cc:68] Stopped Ranger
2026-08-06 12:59:06.097 UTC [4531] LOG: received smart shutdown request
2026-08-06 12:59:06.101 UTC [4531] LOG: background worker "logical replication launcher" (PID 4540) exited with exit code 1
2026-08-06 12:59:06.102 UTC [4535] LOG: shutting down
2026-08-06 12:59:06.252 UTC [4531] LOG: database system is shut down
2026-08-06T12:59:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47700 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:59:06 dist-test-slave-zkpd krb5kdc[6071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:59:06 dist-test-slave-zkpd krb5kdc[6071](info): set up 2 sockets
Aug 06 12:59:06 dist-test-slave-zkpd krb5kdc[6071](info): commencing operation
krb5kdc: starting...
W20260806 12:59:07.424603 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:59:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:59:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:59:07.542577 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45869
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:45869,127.24.83.61:43701,127.24.83.60:42805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:07.621997 6084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:07.622229 6084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:07.622268 6084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:07.623365 6084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:59:07.623432 6084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:07.623456 6084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:07.623476 6084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:59:07.623507 6084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:59:07.624987 6084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:45869,127.24.83.61:43701,127.24.83.60:42805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:07.625257 6084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:07.625488 6084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:07.630177 6089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:07.630177 6092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:07.630187 6090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:07.630460 6084 server_base.cc:1034] running on GCE node
I20260806 12:59:07.630728 6084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:07.631035 6084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:07.632174 6084 hybrid_clock.cc:648] HybridClock initialized: now 1786021147632163 us; error 30 us; skew 500 ppm
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:07.634505 6084 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:59:07.634862 6084 webserver.cc:460] Webserver started at http://127.24.83.62:38271/ using document root <none> and password file <none>
I20260806 12:59:07.635054 6084 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:07.635092 6084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:07.635177 6084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:07.636186 6084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "4cc25a849cb0477b98d9f1e09c923a49"
format_stamp: "Formatted at 2026-08-06 12:59:07 on dist-test-slave-zkpd"
server_key: "c67606bc65a0c621af38fd0799ffdbef"
server_key_iv: "ff2495df480224970cb26a5606258771"
server_key_version: "encryptionkey@0"
I20260806 12:59:07.636540 6084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "4cc25a849cb0477b98d9f1e09c923a49"
format_stamp: "Formatted at 2026-08-06 12:59:07 on dist-test-slave-zkpd"
server_key: "c67606bc65a0c621af38fd0799ffdbef"
server_key_iv: "ff2495df480224970cb26a5606258771"
server_key_version: "encryptionkey@0"
I20260806 12:59:07.637740 6084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:59:07.638408 6099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:07.638579 6084 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:59:07.638684 6084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "4cc25a849cb0477b98d9f1e09c923a49"
format_stamp: "Formatted at 2026-08-06 12:59:07 on dist-test-slave-zkpd"
server_key: "c67606bc65a0c621af38fd0799ffdbef"
server_key_iv: "ff2495df480224970cb26a5606258771"
server_key_version: "encryptionkey@0"
I20260806 12:59:07.638751 6084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:07.665493 6084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:07.667296 6084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:07.667461 6084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:07.671367 6084 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:45869
I20260806 12:59:07.671406 6153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:45869 every 8 connection(s)
I20260806 12:59:07.671754 6084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:59:07.672638 6154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:07.674139 6154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:59:07.677016 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 6084
I20260806 12:59:07.677126 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:59:07.677336 24908 external_mini_cluster.cc:1428] Setting key ec5c2c964f8aec0b8512d72db3d5f1c5
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:07.683851 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.674760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55490 (local address 127.24.83.62:45869)
0806 12:59:07.674905 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:07.674913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:07.675072 (+ 159us) server_negotiation.cc:407] Connection header received
0806 12:59:07.675540 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:07.675542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:07.675557 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:07.675630 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:07.676199 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.677007 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.677984 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.678318 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.678457 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.680582 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:07.680590 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:07.680590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:07.680596 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:07.682205 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.682702 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.682704 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.682705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.682778 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.683063 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.683064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.683064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.683267 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:07.683331 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:07.683564 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 12:59:07.683683 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260806 12:59:07.683954 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.674727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45869 (local address 127.0.0.1:55490)
0806 12:59:07.674959 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.674986 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:07.675418 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:07.675659 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:07.675661 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:07.675952 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:07.676118 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.676122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.677114 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.677115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.677880 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.677891 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.678439 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.678440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.678509 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.679026 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:07.679035 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:07.680501 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:07.682295 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.682303 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.682306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.682620 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.682855 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.682856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.682857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.682981 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.683341 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:07.683348 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:07.683428 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 12:59:07.683538 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 12:59:07.683780 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":66,"thread_start_us":90,"threads_started":1}
I20260806 12:59:07.684684 6154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:59:07.685027 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.684897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:43701 (local address 127.0.0.1:49242)
0806 12:59:07.684947 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.684999 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:43701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 12:59:07.685140 6103 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.83.61:43701: connect: Connection refused (error 111)
I20260806 12:59:07.685557 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.685468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:43701 (local address 127.0.0.1:49254)
0806 12:59:07.685515 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.685540 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:43701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 12:59:07.685684 6154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.61:43701: Network error: Client connection negotiation failed: client connection to 127.24.83.61:43701: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.24.83.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab.
Entry for principal kudu/127.24.83.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab.
I20260806 12:59:07.702828 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.61:43701
--webserver_interface=127.24.83.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab
--principal=kudu/127.24.83.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:45869,127.24.83.61:43701,127.24.83.60:42805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:07.734926 6154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } attempt: 1
I20260806 12:59:07.735373 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.735243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:43701 (local address 127.0.0.1:49258)
0806 12:59:07.735304 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.735344 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:43701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 12:59:07.735703 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.735610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:43701 (local address 127.0.0.1:49262)
0806 12:59:07.735654 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.735678 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:43701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 12:59:07.735796 6154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.61:43701: Network error: Client connection negotiation failed: client connection to 127.24.83.61:43701: connect: Connection refused (error 111)
W20260806 12:59:07.782955 6161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:07.783174 6161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:07.783210 6161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:07.784336 6161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:59:07.784407 6161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:07.784435 6161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:07.784466 6161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:59:07.784490 6161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:59:07.785912 6161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:45869,127.24.83.61:43701,127.24.83.60:42805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab
--principal=kudu/127.24.83.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.61:43701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.83.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:07.786178 6161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:07.786394 6161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:07.791100 6169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:07.791124 6167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:07.791121 6166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:07.791412 6161 server_base.cc:1034] running on GCE node
I20260806 12:59:07.791663 6161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:07.791973 6161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:07.793123 6161 hybrid_clock.cc:648] HybridClock initialized: now 1786021147793102 us; error 38 us; skew 500 ppm
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:07.795611 6161 init.cc:377] Logged in from keytab as kudu/127.24.83.61@KRBTEST.COM (short username kudu)
I20260806 12:59:07.796131 6161 webserver.cc:460] Webserver started at http://127.24.83.61:33483/ using document root <none> and password file <none>
I20260806 12:59:07.796353 6161 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:07.796419 6161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:07.796556 6161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:07.797586 6161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/data/instance:
uuid: "6aa97093c04c4865af1b7763d741af0c"
format_stamp: "Formatted at 2026-08-06 12:59:07 on dist-test-slave-zkpd"
server_key: "b10ee1b4da3c70d2dd79d2976f620f75"
server_key_iv: "1418cd440506ae9ebc39488d14b7ed7f"
server_key_version: "encryptionkey@0"
I20260806 12:59:07.797956 6161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/wal/instance:
uuid: "6aa97093c04c4865af1b7763d741af0c"
format_stamp: "Formatted at 2026-08-06 12:59:07 on dist-test-slave-zkpd"
server_key: "b10ee1b4da3c70d2dd79d2976f620f75"
server_key_iv: "1418cd440506ae9ebc39488d14b7ed7f"
server_key_version: "encryptionkey@0"
I20260806 12:59:07.799304 6161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:59:07.800078 6176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:07.800240 6161 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:59:07.800340 6161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/wal
uuid: "6aa97093c04c4865af1b7763d741af0c"
format_stamp: "Formatted at 2026-08-06 12:59:07 on dist-test-slave-zkpd"
server_key: "b10ee1b4da3c70d2dd79d2976f620f75"
server_key_iv: "1418cd440506ae9ebc39488d14b7ed7f"
server_key_version: "encryptionkey@0"
I20260806 12:59:07.800426 6161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:07.803952 6154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } attempt: 2
I20260806 12:59:07.804332 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.804210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:43701 (local address 127.0.0.1:49268)
0806 12:59:07.804268 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.804300 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:43701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 12:59:07.804651 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.804551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:43701 (local address 127.0.0.1:49284)
0806 12:59:07.804596 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.804620 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:43701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 12:59:07.804738 6154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.61:43701: Network error: Client connection negotiation failed: client connection to 127.24.83.61:43701: connect: Connection refused (error 111)
I20260806 12:59:07.818444 6161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:07.820744 6161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:07.820924 6161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:07.824867 6161 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.61:43701
I20260806 12:59:07.824905 6228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.61:43701 every 8 connection(s)
I20260806 12:59:07.825248 6161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
I20260806 12:59:07.826069 6229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:07.827142 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 6161
I20260806 12:59:07.827224 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-1/wal/instance
I20260806 12:59:07.827430 24908 external_mini_cluster.cc:1428] Setting key 9b24cb9ef0165af8f753f8bd4548255f
I20260806 12:59:07.827728 6229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.83.60@KRBTEST.COM; defaulting to no policy
I20260806 12:59:07.837491 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.828525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55494 (local address 127.24.83.62:45869)
0806 12:59:07.828590 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:07.828592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:07.828772 (+ 180us) server_negotiation.cc:407] Connection header received
0806 12:59:07.829463 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:07.829464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:07.829474 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:07.829524 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:07.830310 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.831055 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.832101 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.832474 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.832555 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.834760 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:07.834761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:07.834762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:07.834769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:07.836283 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.836648 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.836648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.836649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.836724 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.836967 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.836968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.836968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.837086 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:07.837153 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:07.837369 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 12:59:07.837431 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:59:07.837638 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.828485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45869 (local address 127.0.0.1:55494)
0806 12:59:07.828646 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.828688 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:07.829359 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:07.829604 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:07.829608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:07.829951 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:07.830218 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.830223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.831147 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.831148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.832013 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.832016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.832560 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.832561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.832657 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.833127 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:07.833135 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:07.834660 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:07.836365 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.836371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.836373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.836581 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.836794 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.836795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.836795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.836901 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.837157 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:07.837161 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:07.837263 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 12:59:07.837340 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 12:59:07.837454 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.60@KRBTEST.COM" created.
I20260806 12:59:07.838335 6229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
I20260806 12:59:07.846282 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.838562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:43701 (local address 127.0.0.1:49286)
0806 12:59:07.838610 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.838621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:07.838661 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:07.838986 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:07.838986 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:07.839092 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:07.839141 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.839143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.840368 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.840369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.841177 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.841180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.841822 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.841822 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.841875 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.842344 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:07.842349 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:07.843562 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:07.845248 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.845249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.845249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.845417 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.845626 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.845627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.845627 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.845680 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.845968 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:07.845969 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:07.846007 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 12:59:07.846079 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 12:59:07.846172 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 12:59:07.846376 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.838649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49286 (local address 127.24.83.61:43701)
0806 12:59:07.838824 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:07.838829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:07.838854 (+ 25us) server_negotiation.cc:407] Connection header received
0806 12:59:07.838880 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:07.838882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:07.838896 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:07.838973 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:07.839223 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.840281 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.841265 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.841744 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.841819 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.843658 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:07.843667 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:07.843667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:07.843676 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:07.845153 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.845505 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.845508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.845510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.845561 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.845750 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.845751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.845751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.845896 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:07.845989 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:07.846109 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 12:59:07.846239 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260806 12:59:07.846917 6229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:59:07.847273 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.847151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54076)
0806 12:59:07.847199 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.847241 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 12:59:07.847388 6178 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
I20260806 12:59:07.847754 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.847658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54084)
0806 12:59:07.847708 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.847732 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 12:59:07.847862 6229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:42805: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab.
Entry for principal kudu/127.24.83.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab.
I20260806 12:59:07.854005 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.60:42805
--webserver_interface=127.24.83.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab
--principal=kudu/127.24.83.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:45869,127.24.83.61:43701,127.24.83.60:42805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:07.895913 6154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } attempt: 3
I20260806 12:59:07.897082 6229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } attempt: 1
I20260806 12:59:07.897469 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.897360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54088)
0806 12:59:07.897404 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.897431 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 12:59:07.897750 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.897677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54092)
0806 12:59:07.897710 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.897729 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 12:59:07.897889 6229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:42805: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
I20260806 12:59:07.903460 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.896313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:43701 (local address 127.0.0.1:49292)
0806 12:59:07.896370 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.896381 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:07.896435 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:07.896569 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:07.896569 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:07.896735 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:07.896798 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.896800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.897723 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.897725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.898536 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.898539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.898924 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.898925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.899028 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.899467 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:07.899472 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:07.900809 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:07.902374 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.902375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.902376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.902582 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.902791 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.902792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.902793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.902850 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.903211 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:07.903212 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:07.903261 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 12:59:07.903310 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 12:59:07.903379 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 12:59:07.903556 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.896383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49292 (local address 127.24.83.61:43701)
0806 12:59:07.896433 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:07.896436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:07.896446 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:59:07.896499 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:07.896499 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:07.896510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:07.896554 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:07.896892 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.897586 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.898626 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.898847 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.898906 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.900893 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:07.900895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:07.900895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:07.900902 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:07.902274 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.902668 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.902669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.902670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.902727 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.902996 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.902997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.902998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.903142 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:07.903224 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:07.903352 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 12:59:07.903447 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 12:59:07.904099 6154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:59:07.904440 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.904340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54096)
0806 12:59:07.904390 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.904418 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 12:59:07.904717 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.904665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54100)
0806 12:59:07.904686 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.904705 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260806 12:59:07.904812 6154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:42805: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
I20260806 12:59:07.936002 6154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } attempt: 1
I20260806 12:59:07.936472 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.936346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54114)
0806 12:59:07.936405 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.936434 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 12:59:07.936866 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.936760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54126)
0806 12:59:07.936810 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.936844 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 12:59:07.936990 6154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:42805: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
W20260806 12:59:07.938263 6236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:07.938432 6236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:07.938453 6236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:07.939508 6236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:59:07.939543 6236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:07.939558 6236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:07.939570 6236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:59:07.939581 6236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:59:07.941054 6236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:45869,127.24.83.61:43701,127.24.83.60:42805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab
--principal=kudu/127.24.83.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.60:42805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.83.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:07.941300 6236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:07.941596 6236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:07.946432 6241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:07.946424 6244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:07.946496 6236 server_base.cc:1034] running on GCE node
W20260806 12:59:07.946424 6242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:07.946844 6236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:07.947188 6236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:07.948345 6236 hybrid_clock.cc:648] HybridClock initialized: now 1786021147948338 us; error 33 us; skew 500 ppm
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:07.950762 6236 init.cc:377] Logged in from keytab as kudu/127.24.83.60@KRBTEST.COM (short username kudu)
I20260806 12:59:07.951203 6236 webserver.cc:460] Webserver started at http://127.24.83.60:33549/ using document root <none> and password file <none>
I20260806 12:59:07.951390 6236 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:07.951428 6236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:07.951511 6236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:07.952517 6236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/data/instance:
uuid: "8230106e99cd46fca53f542ee7f9a778"
format_stamp: "Formatted at 2026-08-06 12:59:07 on dist-test-slave-zkpd"
server_key: "280555e822fe70cde58804420f6bbc94"
server_key_iv: "59f3153b499c156bef1800884101c7ce"
server_key_version: "encryptionkey@0"
I20260806 12:59:07.952800 6236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/wal/instance:
uuid: "8230106e99cd46fca53f542ee7f9a778"
format_stamp: "Formatted at 2026-08-06 12:59:07 on dist-test-slave-zkpd"
server_key: "280555e822fe70cde58804420f6bbc94"
server_key_iv: "59f3153b499c156bef1800884101c7ce"
server_key_version: "encryptionkey@0"
I20260806 12:59:07.954041 6236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:59:07.954721 6251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:07.954921 6236 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:59:07.955001 6236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/wal
uuid: "8230106e99cd46fca53f542ee7f9a778"
format_stamp: "Formatted at 2026-08-06 12:59:07 on dist-test-slave-zkpd"
server_key: "280555e822fe70cde58804420f6bbc94"
server_key_iv: "59f3153b499c156bef1800884101c7ce"
server_key_version: "encryptionkey@0"
I20260806 12:59:07.955054 6236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:07.963794 6236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:07.965554 6236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:07.965659 6236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:07.966125 6229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } attempt: 2
I20260806 12:59:07.966524 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.966402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54134)
0806 12:59:07.966458 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.966488 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 12:59:07.966915 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.966829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54150)
0806 12:59:07.966875 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.966899 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 12:59:07.967028 6229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:42805: Network error: Client connection negotiation failed: client connection to 127.24.83.60:42805: connect: Connection refused (error 111)
I20260806 12:59:07.969754 6236 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.60:42805
I20260806 12:59:07.969816 6304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.60:42805 every 8 connection(s)
I20260806 12:59:07.970153 6236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
I20260806 12:59:07.970992 6305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:07.972554 6305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:07.978555 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 6236
I20260806 12:59:07.978639 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786020991335182-24908-0/minicluster-data/master-2/wal/instance
I20260806 12:59:07.978863 24908 external_mini_cluster.cc:1428] Setting key 022f7fc208d45ae7cfa22e68254196be
I20260806 12:59:07.981139 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.973220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55502 (local address 127.24.83.62:45869)
0806 12:59:07.973292 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:07.973293 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:07.973455 (+ 162us) server_negotiation.cc:407] Connection header received
0806 12:59:07.973877 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:07.973878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:07.973886 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:07.973941 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:07.974497 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.974968 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.975875 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.976125 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.976179 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.978099 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:07.978101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:07.978101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:07.978109 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:07.979728 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.980183 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.980185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.980185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.980244 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.980506 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.980507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.980508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.980625 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:07.980697 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:07.980974 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 12:59:07.981052 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 12:59:07.981324 6307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.973174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45869 (local address 127.0.0.1:55502)
0806 12:59:07.973328 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.973367 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:07.973800 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:07.974015 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:07.974019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:07.974244 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:07.974428 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.974432 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.975044 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.975045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.975804 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.975805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.976196 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.976197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.976302 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.976714 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:07.976721 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:07.978000 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:07.979837 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.979844 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.979846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.980102 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.980325 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.980326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.980326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.980436 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.980703 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:07.980711 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:07.980849 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 12:59:07.980952 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 12:59:07.981098 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260806 12:59:07.981882 6305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
I20260806 12:59:07.987495 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.979266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.24.83.60:42805)
0806 12:59:07.979453 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:07.979458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:07.979483 (+ 25us) server_negotiation.cc:407] Connection header received
0806 12:59:07.979507 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:07.979509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:07.979522 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:07.979616 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:07.980291 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.980961 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.981961 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.982344 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.982410 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.984670 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:07.984679 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:07.984679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:07.984686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:07.986297 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.986654 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.986663 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.986663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.986711 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.986924 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.986925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.986925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.987067 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:07.987230 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:07.987291 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 12:59:07.987364 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260806 12:59:07.988978 6307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.982125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:43701 (local address 127.0.0.1:49296)
0806 12:59:07.982175 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.982185 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:07.982221 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:07.982323 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:07.982324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:07.982478 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:07.982526 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.982528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.983406 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.983407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.983858 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.983859 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.984257 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.984258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.984331 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.984713 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:07.984718 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:07.986157 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:07.987864 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.987865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.987865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.988111 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.988360 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.988361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.988361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.988418 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.988679 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:07.988680 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:07.988717 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 12:59:07.988798 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 12:59:07.988892 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 12:59:07.988977 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.982178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49296 (local address 127.24.83.61:43701)
0806 12:59:07.982216 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:07.982218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:07.982226 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:59:07.982273 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:07.982274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:07.982282 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:07.982319 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:07.982610 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.983317 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.983979 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.984173 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.984256 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.986245 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:07.986246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:07.986247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:07.986254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:07.987784 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.988180 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.988181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.988181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.988232 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.988486 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.988486 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.988487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.988620 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:07.988674 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:07.988824 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 12:59:07.988892 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 12:59:07.989643 6305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
I20260806 12:59:07.995559 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.988320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55518 (local address 127.24.83.62:45869)
0806 12:59:07.988357 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:07.988359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:07.988383 (+ 24us) server_negotiation.cc:407] Connection header received
0806 12:59:07.988402 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:07.988403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:07.988411 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:07.988460 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:07.988886 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.989510 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.990364 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.990524 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.990566 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.992814 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:07.992815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:07.992815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:07.992822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:07.994485 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.994801 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.994803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.994803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.994860 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.995033 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.995034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.995034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.995125 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:07.995340 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:07.995379 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 12:59:07.995460 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 12:59:07.996028 6307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.989843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54172)
0806 12:59:07.989932 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:07.989940 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:07.989983 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:07.990077 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:07.990078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:07.990168 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:07.990192 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.990192 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.990998 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.990999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.991441 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:07.991443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.991948 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.991950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:07.992003 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.992319 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:07.992324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:07.993329 (+ 1005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:07.995041 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.995041 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.995042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.995178 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.995399 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:07.995399 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:07.995400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:07.995450 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:07.995735 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:07.995736 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:07.995793 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 12:59:07.995865 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 12:59:07.995965 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 12:59:07.996150 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.989900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54172 (local address 127.24.83.60:42805)
0806 12:59:07.989930 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:07.989932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:07.989974 (+ 42us) server_negotiation.cc:407] Connection header received
0806 12:59:07.990017 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:07.990018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:07.990026 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:07.990075 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:07.990250 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.990920 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.991520 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.991840 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.991966 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:07.993410 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:07.993411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:07.993412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:07.993419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:07.994861 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.995268 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.995269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.995270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.995321 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:07.995521 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:07.995522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:07.995523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:07.995648 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:07.995712 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:07.995942 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 12:59:07.996026 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 12:59:07.997319 6305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778: Bootstrap starting.
I20260806 12:59:07.998171 6305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:07.998528 6305 log.cc:824] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:07.999363 6305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778: No bootstrap required, opened a new log
Aug 06 12:59:07 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
I20260806 12:59:08.001030 6305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } }
I20260806 12:59:08.001163 6305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:08.001209 6305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8230106e99cd46fca53f542ee7f9a778, State: Initialized, Role: FOLLOWER
I20260806 12:59:08.001353 6305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } }
I20260806 12:59:08.001652 6305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:59:08.001721 6311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } } }
I20260806 12:59:08.001818 6311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:59:08.003767 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:07.996202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49304 (local address 127.24.83.61:43701)
0806 12:59:07.996414 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:07.996416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:07.996424 (+ 8us) server_negotiation.cc:407] Connection header received
0806 12:59:07.996443 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:07.996443 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:07.996451 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:07.996525 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:07.996983 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.997435 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.998057 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:07.998270 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:07.998331 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:08.000667 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:08.000668 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:08.000669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:08.000676 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:08.002262 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.002673 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.002673 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.002674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.002720 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.002989 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.002990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.002990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.003130 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:08.003326 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:08.003568 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 12:59:08.003667 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
W20260806 12:59:08.003703 6322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:59:08.004204 6322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:59:08.005129 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:59:08.004984 6236 master_runner.cc:186] Error getting master registration for 127.24.83.62:45869: OK, instance_id { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" instance_seqno: 1786021147670299 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 12:59:08.007332 6236 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 12:59:08.012128 6154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } attempt: 2
I20260806 12:59:08.014354 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:08.006458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54178 (local address 127.24.83.60:42805)
0806 12:59:08.006492 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:08.006494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:08.006895 (+ 401us) server_negotiation.cc:407] Connection header received
0806 12:59:08.006954 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:08.006955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:08.006966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:08.007021 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:08.007741 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.008495 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.009496 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.009806 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.009848 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:08.011260 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:08.011262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:08.011262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:08.011281 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:08.013005 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.013428 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.013429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.013429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.013484 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.013703 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.013704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.013704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.013871 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:08.013957 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:08.014174 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 12:59:08.014252 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 12:59:08.015606 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:08.006772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.24.83.61:43701)
0806 12:59:08.006809 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:08.006811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:08.007154 (+ 343us) server_negotiation.cc:407] Connection header received
0806 12:59:08.007552 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:08.007553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:08.007563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:08.007592 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:08.008305 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.009019 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.009950 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.010219 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.010256 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:08.011688 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:08.011690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:08.011690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:08.011696 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:08.013889 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.014284 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.014286 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.014286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.014341 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.014823 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.014825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.014825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.014957 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:08.015049 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:08.015430 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 12:59:08.015503 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 12:59:08.016670 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:08.006448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55526 (local address 127.24.83.62:45869)
0806 12:59:08.006487 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:08.006489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:08.007042 (+ 553us) server_negotiation.cc:407] Connection header received
0806 12:59:08.007403 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:08.007405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:08.007418 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:08.007453 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:08.008589 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.009258 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.010382 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.010944 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.010990 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:08.013540 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:08.013541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:08.013542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:08.013548 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:08.015458 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.015851 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.015853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.015853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.015934 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.016132 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.016134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.016134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.016258 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:08.016345 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:08.016505 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 12:59:08.016590 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 06 12:59:08 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
I20260806 12:59:08.021274 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:08.012598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.24.83.60:42805)
0806 12:59:08.014485 (+ 1887us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:08.014486 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:08.014493 (+ 7us) server_negotiation.cc:407] Connection header received
0806 12:59:08.014511 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:08.014512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:08.014520 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:08.014582 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:08.014903 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.015553 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.016812 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.017102 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.017190 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:08.018875 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:08.018876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:08.018877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:08.018883 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:08.020315 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.020632 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.020633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.020633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.020687 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.020844 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.020845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.020845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.020956 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:08.021024 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:08.021112 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 12:59:08.021180 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1874,"thread_start_us":90,"threads_started":1}
I20260806 12:59:08.021301 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:08.012369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54194)
0806 12:59:08.012465 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:08.012475 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:08.012509 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:08.014589 (+ 2080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:08.014590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:08.014764 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:08.014816 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:08.014818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.015644 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.015645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:08.016412 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:08.016414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.017191 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.017192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:08.017257 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:08.017650 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:08.017654 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:08.018810 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:08.020386 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:08.020387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:08.020387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:08.020564 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:08.020748 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:08.020749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:08.020749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:08.020793 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:08.021012 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:08.021013 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:08.021043 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 12:59:08.021090 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 12:59:08.021152 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 12:59:08.022774 6154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49: Bootstrap starting.
I20260806 12:59:08.023478 6154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:08.023767 6154 log.cc:824] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:08.024672 6154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49: No bootstrap required, opened a new log
I20260806 12:59:08.026676 6154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } }
I20260806 12:59:08.026816 6154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:08.026854 6154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc25a849cb0477b98d9f1e09c923a49, State: Initialized, Role: FOLLOWER
I20260806 12:59:08.026974 6154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } }
I20260806 12:59:08.027215 6333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } } }
I20260806 12:59:08.027287 6154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:59:08.027307 6333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 12:59:08.029109 6344 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:59:08.029209 6344 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:59:08.029735 6084 master_runner.cc:186] Error getting master registration for 127.24.83.61:43701: OK, instance_id { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" instance_seqno: 1786021147823788 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 12:59:08.031862 6084 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 12:59:08.058172 6229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } attempt: 3
Aug 06 12:59:08 dist-test-slave-zkpd krb5kdc[6071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
I20260806 12:59:08.065351 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:08.058524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54208 (local address 127.24.83.60:42805)
0806 12:59:08.058568 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:08.058570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:08.058579 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:59:08.058661 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:08.058662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:08.058679 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:08.058746 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:08.059071 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.059566 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.060297 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.060491 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.060565 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:08.062394 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:08.062395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:08.062396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:08.062404 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:08.063988 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.064519 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.064520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.064520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.064589 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:08.064809 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:08.064810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:08.064810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:08.064932 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:08.065004 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:08.065178 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 12:59:08.065250 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:59:08.065351 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:08.058475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:42805 (local address 127.0.0.1:54208)
0806 12:59:08.058526 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:08.058537 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:08.058578 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:08.058776 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:08.058777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:08.058944 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:08.058987 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:08.058989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.059655 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.059657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:08.060211 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:08.060213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:08.060575 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:08.060576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:08.060639 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:08.061126 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:08.061138 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:08.062303 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:08.064215 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:08.064215 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:08.064216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:08.064436 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:08.064668 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:08.064669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:08.064669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:08.064743 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:08.065009 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:08.065010 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:08.065057 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 12:59:08.065137 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 12:59:08.065250 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 12:59:08.067004 6229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c: Bootstrap starting.
I20260806 12:59:08.067654 6229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:08.068032 6229 log.cc:824] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:08.068784 6229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c: No bootstrap required, opened a new log
I20260806 12:59:08.070214 6229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } }
I20260806 12:59:08.070358 6229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:08.070411 6229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aa97093c04c4865af1b7763d741af0c, State: Initialized, Role: FOLLOWER
I20260806 12:59:08.070539 6229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } }
I20260806 12:59:08.070807 6347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } } }
I20260806 12:59:08.070869 6229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:59:08.071038 6347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [sys.catalog]: This master's current role is: FOLLOWER
W20260806 12:59:08.071527 6182 tablet.cc:2369] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:59:08.072844 6358 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:59:08.072966 6358 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:59:08.075436 6161 master_runner.cc:263] No leader master found from master 6aa97093c04c4865af1b7763d741af0c
I20260806 12:59:08.075486 6161 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 12:59:08.107033 6347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:59:08.107156 6347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } }
I20260806 12:59:08.107388 6347 leader_election.cc:288] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cc25a849cb0477b98d9f1e09c923a49 (127.24.83.62:45869), 8230106e99cd46fca53f542ee7f9a778 (127.24.83.60:42805)
I20260806 12:59:08.107802 6279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6aa97093c04c4865af1b7763d741af0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8230106e99cd46fca53f542ee7f9a778" is_pre_election: true
I20260806 12:59:08.107975 6279 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6aa97093c04c4865af1b7763d741af0c in term 0.
I20260806 12:59:08.108301 6178 leader_election.cc:302] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6aa97093c04c4865af1b7763d741af0c, 8230106e99cd46fca53f542ee7f9a778; no voters:
I20260806 12:59:08.108479 6347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:59:08.108528 6347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:59:08.108551 6347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:08.109472 6347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } }
I20260806 12:59:08.109577 6347 leader_election.cc:288] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [CANDIDATE]: Term 1 election: Requested vote from peers 4cc25a849cb0477b98d9f1e09c923a49 (127.24.83.62:45869), 8230106e99cd46fca53f542ee7f9a778 (127.24.83.60:42805)
I20260806 12:59:08.109989 6126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6aa97093c04c4865af1b7763d741af0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cc25a849cb0477b98d9f1e09c923a49"
I20260806 12:59:08.110105 6126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:08.110913 6126 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6aa97093c04c4865af1b7763d741af0c in term 1.
I20260806 12:59:08.111138 6127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6aa97093c04c4865af1b7763d741af0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cc25a849cb0477b98d9f1e09c923a49" is_pre_election: true
I20260806 12:59:08.111222 6127 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6aa97093c04c4865af1b7763d741af0c in term 1. Re-sending same reply.
I20260806 12:59:08.111434 6279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6aa97093c04c4865af1b7763d741af0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8230106e99cd46fca53f542ee7f9a778"
I20260806 12:59:08.111502 6279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:08.112036 6180 leader_election.cc:302] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cc25a849cb0477b98d9f1e09c923a49, 6aa97093c04c4865af1b7763d741af0c; no voters:
I20260806 12:59:08.112676 6279 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6aa97093c04c4865af1b7763d741af0c in term 1.
I20260806 12:59:08.112914 6347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:59:08.113224 6347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [term 1 LEADER]: Becoming Leader. State: Replica: 6aa97093c04c4865af1b7763d741af0c, State: Running, Role: LEADER
I20260806 12:59:08.113317 6347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } }
I20260806 12:59:08.113653 6347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aa97093c04c4865af1b7763d741af0c. Latest consensus state: current_term: 1 leader_uuid: "6aa97093c04c4865af1b7763d741af0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } } }
I20260806 12:59:08.113739 6347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:08.114027 6362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:59:08.114240 6362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:59:08.116165 6279 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [term 1 FOLLOWER]: Refusing update from remote peer 6aa97093c04c4865af1b7763d741af0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:08.116513 6347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:08.117774 6311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aa97093c04c4865af1b7763d741af0c. Latest consensus state: current_term: 1 leader_uuid: "6aa97093c04c4865af1b7763d741af0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } } }
I20260806 12:59:08.117872 6311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:59:08.119026 6127 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [term 1 FOLLOWER]: Refusing update from remote peer 6aa97093c04c4865af1b7763d741af0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:08.119331 6347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:08.120584 6333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aa97093c04c4865af1b7763d741af0c. Latest consensus state: current_term: 1 leader_uuid: "6aa97093c04c4865af1b7763d741af0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } } }
I20260806 12:59:08.120667 6333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:59:08.122418 6363 mvcc.cc:205] Tried to move back new op lower bound from 7315542622681509888 to 7315542622672535552. Current Snapshot: MvccSnapshot[applied={T|T < 7315542622681509888}]
I20260806 12:59:08.122754 6347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6aa97093c04c4865af1b7763d741af0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } } }
I20260806 12:59:08.122826 6347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:08.123335 6333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6aa97093c04c4865af1b7763d741af0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } } }
I20260806 12:59:08.123409 6333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:59:08.123726 6366 mvcc.cc:205] Tried to move back new op lower bound from 7315542622681509888 to 7315542622672535552. Current Snapshot: MvccSnapshot[applied={T|T < 7315542622681509888}]
I20260806 12:59:08.124127 6347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6aa97093c04c4865af1b7763d741af0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } } }
I20260806 12:59:08.124208 6347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:08.124589 6311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6aa97093c04c4865af1b7763d741af0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc25a849cb0477b98d9f1e09c923a49" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45869 } } peers { permanent_uuid: "6aa97093c04c4865af1b7763d741af0c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 43701 } } peers { permanent_uuid: "8230106e99cd46fca53f542ee7f9a778" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 42805 } } }
I20260806 12:59:08.124666 6311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:59:08.125316 6364 mvcc.cc:205] Tried to move back new op lower bound from 7315542622681509888 to 7315542622672535552. Current Snapshot: MvccSnapshot[applied={T|T < 7315542622681509888}]
I20260806 12:59:08.126144 6362 catalog_manager.cc:1347] Generated new cluster ID: af3b67d7010b4d68b9315ae9e275eceb
I20260806 12:59:08.126202 6362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:59:08.143710 6362 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:59:08.144546 6362 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:59:08.149271 6362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6aa97093c04c4865af1b7763d741af0c: Generated new TSK 0
I20260806 12:59:08.149524 6362 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:59:09.004662 6322 catalog_manager.cc:1259] Loaded cluster ID: af3b67d7010b4d68b9315ae9e275eceb
I20260806 12:59:09.004753 6322 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778: loading cluster ID for follower catalog manager: success
I20260806 12:59:09.005615 6322 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778: acquiring CA information for follower catalog manager: success
I20260806 12:59:09.005919 6322 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8230106e99cd46fca53f542ee7f9a778: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 12:59:09.029683 6344 catalog_manager.cc:1259] Loaded cluster ID: af3b67d7010b4d68b9315ae9e275eceb
I20260806 12:59:09.029770 6344 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49: loading cluster ID for follower catalog manager: success
I20260806 12:59:09.030566 6344 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49: acquiring CA information for follower catalog manager: success
I20260806 12:59:09.030879 6344 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4cc25a849cb0477b98d9f1e09c923a49: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 12:59:09.207405 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 6084
I20260806 12:59:09.213316 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 6161
I20260806 12:59:09.219120 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 6236
2026-08-06T12:59:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2850 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:59:09 dist-test-slave-zkpd krb5kdc[6378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:59:09 dist-test-slave-zkpd krb5kdc[6378](info): set up 2 sockets
Aug 06 12:59:09 dist-test-slave-zkpd krb5kdc[6378](info): commencing operation
krb5kdc: starting...
W20260806 12:59:10.268579 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:59:10 dist-test-slave-zkpd krb5kdc[6378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:59:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:59:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:59:10.390763 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:34329
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:44375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:34329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:10.472637 6391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:10.472846 6391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:10.472867 6391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:10.473948 6391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:59:10.473984 6391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:10.473997 6391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:10.474009 6391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:59:10.474021 6391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:59:10.475380 6391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:44375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:34329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:34329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:10.475565 6391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:10.475778 6391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:10.480640 6397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:10.480634 6399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:10.480652 6396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:10.480767 6391 server_base.cc:1034] running on GCE node
I20260806 12:59:10.481493 6391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:10.481782 6391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:10.482945 6391 hybrid_clock.cc:648] HybridClock initialized: now 1786021150482935 us; error 47 us; skew 500 ppm
Aug 06 12:59:10 dist-test-slave-zkpd krb5kdc[6378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021150, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:10.485527 6391 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:59:10.485900 6391 webserver.cc:460] Webserver started at http://127.24.83.62:41731/ using document root <none> and password file <none>
I20260806 12:59:10.486073 6391 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:10.486110 6391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:10.486198 6391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:10.487171 6391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "91abcdf5cf224e01bfe3a3e1ec113435"
format_stamp: "Formatted at 2026-08-06 12:59:10 on dist-test-slave-zkpd"
server_key: "a8d8e2319735d004139510d711b0a3ea"
server_key_iv: "8725d2d03830caf232d0e99ac1360491"
server_key_version: "encryptionkey@0"
I20260806 12:59:10.487473 6391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "91abcdf5cf224e01bfe3a3e1ec113435"
format_stamp: "Formatted at 2026-08-06 12:59:10 on dist-test-slave-zkpd"
server_key: "a8d8e2319735d004139510d711b0a3ea"
server_key_iv: "8725d2d03830caf232d0e99ac1360491"
server_key_version: "encryptionkey@0"
I20260806 12:59:10.488807 6391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:59:10.489547 6406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:10.489796 6391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:59:10.489892 6391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "91abcdf5cf224e01bfe3a3e1ec113435"
format_stamp: "Formatted at 2026-08-06 12:59:10 on dist-test-slave-zkpd"
server_key: "a8d8e2319735d004139510d711b0a3ea"
server_key_iv: "8725d2d03830caf232d0e99ac1360491"
server_key_version: "encryptionkey@0"
I20260806 12:59:10.489949 6391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:10.511013 6391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:10.512866 6391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:10.512976 6391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:10.516750 6391 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:34329
I20260806 12:59:10.516803 6458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:34329 every 8 connection(s)
I20260806 12:59:10.517153 6391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:59:10.518050 6459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:10.520423 6459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435: Bootstrap starting.
I20260806 12:59:10.520987 6459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:10.521276 6459 log.cc:824] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:10.521986 6459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435: No bootstrap required, opened a new log
I20260806 12:59:10.523792 6459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91abcdf5cf224e01bfe3a3e1ec113435" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34329 } }
I20260806 12:59:10.523973 6459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:10.524021 6459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91abcdf5cf224e01bfe3a3e1ec113435, State: Initialized, Role: FOLLOWER
I20260806 12:59:10.524183 6459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91abcdf5cf224e01bfe3a3e1ec113435" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34329 } }
I20260806 12:59:10.524274 6459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:59:10.524319 6459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:59:10.524369 6459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:10.524878 6459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91abcdf5cf224e01bfe3a3e1ec113435" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34329 } }
I20260806 12:59:10.524999 6459 leader_election.cc:302] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91abcdf5cf224e01bfe3a3e1ec113435; no voters:
I20260806 12:59:10.525178 6459 leader_election.cc:288] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:59:10.525199 6462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:59:10.525396 6462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [term 1 LEADER]: Becoming Leader. State: Replica: 91abcdf5cf224e01bfe3a3e1ec113435, State: Running, Role: LEADER
I20260806 12:59:10.525538 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 6391
I20260806 12:59:10.525435 6459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:59:10.525594 6462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91abcdf5cf224e01bfe3a3e1ec113435" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34329 } }
I20260806 12:59:10.525666 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:59:10.525909 24908 external_mini_cluster.cc:1428] Setting key 82f2c81bbd1ffa2e39bf3afd3b9a89c0
I20260806 12:59:10.526088 6464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91abcdf5cf224e01bfe3a3e1ec113435. Latest consensus state: current_term: 1 leader_uuid: "91abcdf5cf224e01bfe3a3e1ec113435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91abcdf5cf224e01bfe3a3e1ec113435" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34329 } } }
I20260806 12:59:10.526185 6464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:10.526216 6463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91abcdf5cf224e01bfe3a3e1ec113435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91abcdf5cf224e01bfe3a3e1ec113435" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34329 } } }
I20260806 12:59:10.526304 6463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:10.526865 6467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:59:10.527456 6467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:59:10.529098 6467 catalog_manager.cc:1347] Generated new cluster ID: 44bc6d93ce874622b965867698e62ebf
I20260806 12:59:10.529150 6467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:59:10 dist-test-slave-zkpd krb5kdc[6378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:10.536904 6475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:10.526620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32992 (local address 127.24.83.62:34329)
0806 12:59:10.527084 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:10.527093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:10.527130 (+ 37us) server_negotiation.cc:407] Connection header received
0806 12:59:10.527248 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:10.527255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:10.527312 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:10.527451 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:10.528167 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:10.529415 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:10.530197 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:10.530624 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:10.530743 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:10.533099 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:10.533105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:10.533106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:10.533122 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:10.535307 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:10.535682 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:10.535686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:10.535686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:10.535793 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:10.536019 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:10.536020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:10.536020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:10.536286 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:10.536454 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:10.536560 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 12:59:10.536700 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":75,"threads_started":1}
I20260806 12:59:10.537878 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:59:10.552628 6467 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:59:10.553154 6467 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:59:10.562304 6467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91abcdf5cf224e01bfe3a3e1ec113435: Generated new TSK 0
I20260806 12:59:10.562506 6467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:59:10.591621 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 6391
2026-08-06T12:59:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1370 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:59:10 dist-test-slave-zkpd krb5kdc[6488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:59:10 dist-test-slave-zkpd krb5kdc[6488](info): set up 2 sockets
Aug 06 12:59:10 dist-test-slave-zkpd krb5kdc[6488](info): commencing operation
krb5kdc: starting...
W20260806 12:59:11.636480 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:59:11 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:59:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:59:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:59:11.755440 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33939
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:33939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:11.837275 6502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:11.837489 6502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:11.837512 6502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:11.838658 6502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:59:11.838702 6502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:11.838716 6502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:11.838730 6502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:59:11.838742 6502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:59:11.840323 6502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:33939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:11.840579 6502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:11.840809 6502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:11.845767 6510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:11.845849 6502 server_base.cc:1034] running on GCE node
W20260806 12:59:11.845732 6508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:11.845732 6507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:11.846256 6502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:11.846578 6502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:11.847779 6502 hybrid_clock.cc:648] HybridClock initialized: now 1786021151847758 us; error 39 us; skew 500 ppm
Aug 06 12:59:11 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021151, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:11.850297 6502 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:59:11.850800 6502 webserver.cc:460] Webserver started at http://127.24.83.62:36633/ using document root <none> and password file <none>
I20260806 12:59:11.851042 6502 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:11.851117 6502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:11.851269 6502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:11.852797 6502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "478b4505ddc5406595f298298d9ce226"
format_stamp: "Formatted at 2026-08-06 12:59:11 on dist-test-slave-zkpd"
server_key: "2a8393524e63944bde2e66468bad61b6"
server_key_iv: "5af0b83a053c3056405ba267afd323f0"
server_key_version: "encryptionkey@0"
I20260806 12:59:11.853178 6502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "478b4505ddc5406595f298298d9ce226"
format_stamp: "Formatted at 2026-08-06 12:59:11 on dist-test-slave-zkpd"
server_key: "2a8393524e63944bde2e66468bad61b6"
server_key_iv: "5af0b83a053c3056405ba267afd323f0"
server_key_version: "encryptionkey@0"
I20260806 12:59:11.854538 6502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:59:11.855278 6517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:11.855453 6502 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:59:11.855561 6502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "478b4505ddc5406595f298298d9ce226"
format_stamp: "Formatted at 2026-08-06 12:59:11 on dist-test-slave-zkpd"
server_key: "2a8393524e63944bde2e66468bad61b6"
server_key_iv: "5af0b83a053c3056405ba267afd323f0"
server_key_version: "encryptionkey@0"
I20260806 12:59:11.855635 6502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:11.885344 6502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:11.887271 6502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:11.887454 6502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:11.891752 6502 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:33939
I20260806 12:59:11.891788 6569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:33939 every 8 connection(s)
I20260806 12:59:11.892192 6502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:59:11.893172 6570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:11.895496 6570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Bootstrap starting.
I20260806 12:59:11.896122 6570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:11.896416 6570 log.cc:824] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:11.897112 6570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: No bootstrap required, opened a new log
I20260806 12:59:11.898734 6570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:11.898872 6570 raft_consensus.cc:374] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:11.898927 6570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478b4505ddc5406595f298298d9ce226, State: Initialized, Role: FOLLOWER
I20260806 12:59:11.899083 6570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:11.899147 6570 raft_consensus.cc:388] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:59:11.899187 6570 raft_consensus.cc:482] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:59:11.899237 6570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:11.900126 6570 raft_consensus.cc:504] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:11.900236 6570 leader_election.cc:302] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 478b4505ddc5406595f298298d9ce226; no voters:
I20260806 12:59:11.900411 6570 leader_election.cc:288] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:59:11.900456 6574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:59:11.900624 6574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 1 LEADER]: Becoming Leader. State: Replica: 478b4505ddc5406595f298298d9ce226, State: Running, Role: LEADER
I20260806 12:59:11.900631 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 6502
I20260806 12:59:11.900701 6570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:59:11.900740 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:59:11.900713 6574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:11.900972 24908 external_mini_cluster.cc:1428] Setting key 00a9b9786449be61f4044c6ca1874b9c
I20260806 12:59:11.901089 6575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "478b4505ddc5406595f298298d9ce226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } } }
I20260806 12:59:11.901171 6575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:11.901081 6574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 478b4505ddc5406595f298298d9ce226. Latest consensus state: current_term: 1 leader_uuid: "478b4505ddc5406595f298298d9ce226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } } }
I20260806 12:59:11.901528 6579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:59:11.901558 6574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:11.902285 6579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:59:11.904803 6579 catalog_manager.cc:1347] Generated new cluster ID: 7559899162e9487f8050f0a3ac061f6d
I20260806 12:59:11.904850 6579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:59:11 dist-test-slave-zkpd krb5kdc[6488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:11.913722 6585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:11.901667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51544 (local address 127.24.83.62:33939)
0806 12:59:11.902078 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:11.902087 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:11.902115 (+ 28us) server_negotiation.cc:407] Connection header received
0806 12:59:11.902230 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:11.902237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:11.902316 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:11.902480 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:11.903370 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:11.904830 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:11.905884 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:11.906544 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:11.906675 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:11.909464 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:11.909468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:11.909468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:11.909480 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:11.912065 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:11.912466 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:11.912469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:11.912470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:11.912602 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:11.912813 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:11.912814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:11.912814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:11.913102 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:11.913235 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:11.913361 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 12:59:11.913486 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":78,"threads_started":1}
I20260806 12:59:11.921454 6579 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:59:11.922101 6579 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:59:11.926002 6579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Generated new TSK 0
I20260806 12:59:11.926203 6579 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:59:11.954185 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33939
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:12.036316 6597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:12.036559 6597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:12.036599 6597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:12.037760 6597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:12.037832 6597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:12.037876 6597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:59:12.039363 6597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:12.039635 6597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:12.039873 6597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:12.044715 6602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:12.044734 6605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:12.045069 6603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:12.045809 6597 server_base.cc:1034] running on GCE node
I20260806 12:59:12.045995 6597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:12.046195 6597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:12.047334 6597 hybrid_clock.cc:648] HybridClock initialized: now 1786021152047321 us; error 30 us; skew 500 ppm
Aug 06 12:59:12 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:12.049937 6597 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:59:12.050304 6597 webserver.cc:460] Webserver started at http://127.24.83.1:44733/ using document root <none> and password file <none>
I20260806 12:59:12.050483 6597 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:12.050522 6597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:12.050608 6597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:12.051620 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "dc507127e3f64fd19e6c814055a0e35b"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "f8a98ec0b1c820f9ebd839bd5dcc33b0"
server_key_iv: "34945ae537ac38b298de40f0d9d35964"
server_key_version: "encryptionkey@0"
I20260806 12:59:12.051985 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "dc507127e3f64fd19e6c814055a0e35b"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "f8a98ec0b1c820f9ebd839bd5dcc33b0"
server_key_iv: "34945ae537ac38b298de40f0d9d35964"
server_key_version: "encryptionkey@0"
I20260806 12:59:12.053233 6597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:12.053900 6612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:12.054067 6597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:59:12.054173 6597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "dc507127e3f64fd19e6c814055a0e35b"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "f8a98ec0b1c820f9ebd839bd5dcc33b0"
server_key_iv: "34945ae537ac38b298de40f0d9d35964"
server_key_version: "encryptionkey@0"
I20260806 12:59:12.054247 6597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:12.062901 6597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:12.065137 6597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:12.065291 6597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:12.065513 6597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:12.065805 6597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:12.065860 6597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:12.065905 6597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:12.065943 6597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:12.071730 6597 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:44351
I20260806 12:59:12.071846 6727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:44351 every 8 connection(s)
I20260806 12:59:12.072158 6597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
Aug 06 12:59:12 dist-test-slave-zkpd krb5kdc[6488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:12.079411 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 6597
I20260806 12:59:12.079501 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:59:12.079721 24908 external_mini_cluster.cc:1428] Setting key d283a4ea9be20ad3c1f2139777e6199a
I20260806 12:59:12.081090 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.072859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:57813)
0806 12:59:12.073019 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:12.073059 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:12.073535 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:12.073747 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:12.073750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:12.074289 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:12.074467 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.074472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.075124 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.075125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.075987 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.075990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.076334 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.076335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.076434 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.076849 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:12.076857 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:12.078166 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:12.079638 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:12.079646 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:12.079648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:12.079952 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:12.080197 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:12.080197 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:12.080198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:12.080326 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:12.080624 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:12.080629 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:12.080738 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 12:59:12.080820 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 12:59:12.080939 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260806 12:59:12.081115 6585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.072896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:57813 (local address 127.24.83.62:33939)
0806 12:59:12.072958 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:12.072960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:12.073153 (+ 193us) server_negotiation.cc:407] Connection header received
0806 12:59:12.073616 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:12.073618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:12.073629 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:12.073693 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:12.074546 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.075044 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.076070 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.076259 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.076314 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.078264 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:12.078266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:12.078266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:12.078273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:12.079573 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:12.080050 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:12.080051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:12.080052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:12.080117 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:12.080398 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:12.080399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:12.080399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:12.080549 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:12.080627 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:12.080870 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 12:59:12.080974 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:59:12.081722 6728 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33939
I20260806 12:59:12.081838 6728 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:12.082041 6728 heartbeater.cc:511] Master 127.24.83.62:33939 requested a full tablet report, sending...
I20260806 12:59:12.082540 6534 ts_manager.cc:194] Registered new tserver with Master: dc507127e3f64fd19e6c814055a0e35b (127.24.83.1:44351)
I20260806 12:59:12.083137 6534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:57813
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:59:12.106509 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33939
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:12.188751 6733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:12.188966 6733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:12.188990 6733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:12.190063 6733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:12.190095 6733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:12.190120 6733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:59:12.191602 6733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:12.191800 6733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:12.191992 6733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:12.196969 6738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:12.197039 6739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:12.197124 6733 server_base.cc:1034] running on GCE node
W20260806 12:59:12.197255 6741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:12.197496 6733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:12.197793 6733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:12.198956 6733 hybrid_clock.cc:648] HybridClock initialized: now 1786021152198951 us; error 39 us; skew 500 ppm
Aug 06 12:59:12 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:12.201699 6733 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:59:12.202253 6733 webserver.cc:460] Webserver started at http://127.24.83.2:38135/ using document root <none> and password file <none>
I20260806 12:59:12.202503 6733 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:12.202572 6733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:12.202709 6733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:12.203760 6733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "d24eebe04e534ca6b37caf56fae9bbd7"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "cf14a3fd2f663b64adc2057bc3ddb81c"
server_key_iv: "f898fdfbf02c4d59c6c86c24e9ec2a66"
server_key_version: "encryptionkey@0"
I20260806 12:59:12.204159 6733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "d24eebe04e534ca6b37caf56fae9bbd7"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "cf14a3fd2f663b64adc2057bc3ddb81c"
server_key_iv: "f898fdfbf02c4d59c6c86c24e9ec2a66"
server_key_version: "encryptionkey@0"
I20260806 12:59:12.205493 6733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:12.206200 6748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:12.206377 6733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:59:12.206482 6733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "d24eebe04e534ca6b37caf56fae9bbd7"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "cf14a3fd2f663b64adc2057bc3ddb81c"
server_key_iv: "f898fdfbf02c4d59c6c86c24e9ec2a66"
server_key_version: "encryptionkey@0"
I20260806 12:59:12.206565 6733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:12.240181 6733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:12.242429 6733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:12.242576 6733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:12.242832 6733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:12.243227 6733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:12.243280 6733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:12.243314 6733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:12.243333 6733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:12.249493 6733 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:44315
I20260806 12:59:12.249539 6863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:44315 every 8 connection(s)
I20260806 12:59:12.249861 6733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:59:12.251850 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 6733
I20260806 12:59:12.251994 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:59:12.252198 24908 external_mini_cluster.cc:1428] Setting key e53e89d7054c114e87e82f51e9f79236
Aug 06 12:59:12 dist-test-slave-zkpd krb5kdc[6488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:12.260706 6585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.250599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:45031 (local address 127.24.83.62:33939)
0806 12:59:12.250682 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:12.250684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:12.250869 (+ 185us) server_negotiation.cc:407] Connection header received
0806 12:59:12.251366 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:12.251367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:12.251385 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:12.251437 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:12.252607 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.253323 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.254350 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.254620 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.254678 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.257469 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:12.257471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:12.257471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:12.257478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:12.259269 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:12.259656 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:12.259657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:12.259658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:12.259725 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:12.260025 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:12.260026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:12.260027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:12.260176 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:12.260255 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:12.260516 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 12:59:12.260608 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 12:59:12.260735 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.250558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:45031)
0806 12:59:12.250739 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:12.250778 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:12.251257 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:12.251493 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:12.251496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:12.252209 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:12.252516 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.252523 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.253419 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.253420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.254252 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.254255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.254706 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.254707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.254869 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.255425 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:12.255436 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:12.257375 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:12.259347 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:12.259354 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:12.259356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:12.259576 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:12.259802 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:12.259802 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:12.259803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:12.259961 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:12.260254 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:12.260260 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:12.260375 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 12:59:12.260451 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 12:59:12.260570 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260806 12:59:12.261436 6864 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33939
I20260806 12:59:12.261557 6864 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:12.261761 6864 heartbeater.cc:511] Master 127.24.83.62:33939 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
I20260806 12:59:12.262178 6534 ts_manager.cc:194] Registered new tserver with Master: d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315)
I20260806 12:59:12.262569 6534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:45031
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:59:12.279392 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33939
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:12.363163 6869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:12.363391 6869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:12.363430 6869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:12.364578 6869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:12.364645 6869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:12.364691 6869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:59:12.366173 6869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:33939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:12.366454 6869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:12.366686 6869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:12.371682 6874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:12.371713 6875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:12.371713 6869 server_base.cc:1034] running on GCE node
W20260806 12:59:12.371695 6877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:12.372159 6869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:12.372426 6869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:12.373601 6869 hybrid_clock.cc:648] HybridClock initialized: now 1786021152373586 us; error 45 us; skew 500 ppm
Aug 06 12:59:12 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:12.376192 6869 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:59:12.376621 6869 webserver.cc:460] Webserver started at http://127.24.83.3:42625/ using document root <none> and password file <none>
I20260806 12:59:12.376873 6869 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:12.376943 6869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:12.377090 6869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:12.378139 6869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "b66d672365434791a0d2d43d60ce91db"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "53bf9b5073132ffbe29a16c2c0c84c56"
server_key_iv: "b26639998f0b4728fd30ad32df02b9df"
server_key_version: "encryptionkey@0"
I20260806 12:59:12.378505 6869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "b66d672365434791a0d2d43d60ce91db"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "53bf9b5073132ffbe29a16c2c0c84c56"
server_key_iv: "b26639998f0b4728fd30ad32df02b9df"
server_key_version: "encryptionkey@0"
I20260806 12:59:12.379881 6869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260806 12:59:12.380695 6884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:12.380874 6869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:59:12.380975 6869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "b66d672365434791a0d2d43d60ce91db"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "53bf9b5073132ffbe29a16c2c0c84c56"
server_key_iv: "b26639998f0b4728fd30ad32df02b9df"
server_key_version: "encryptionkey@0"
I20260806 12:59:12.381058 6869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:12.413666 6869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:12.415619 6869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:12.415792 6869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:12.416059 6869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:12.416399 6869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:12.416455 6869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:12.416497 6869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:12.416525 6869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:12.422245 6869 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:35745
I20260806 12:59:12.422299 6998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:35745 every 8 connection(s)
I20260806 12:59:12.422612 6869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:59:12.424602 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 6869
I20260806 12:59:12.424670 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:59:12.424857 24908 external_mini_cluster.cc:1428] Setting key 7995b17a593905d1c8b03ce8eae2667c
Aug 06 12:59:12 dist-test-slave-zkpd krb5kdc[6488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:12.431974 6585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.423367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:56081 (local address 127.24.83.62:33939)
0806 12:59:12.423454 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:12.423456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:12.423671 (+ 215us) server_negotiation.cc:407] Connection header received
0806 12:59:12.424242 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:12.424244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:12.424254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:12.424308 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:12.425370 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.425917 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.426980 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.427207 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.427281 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.429384 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:12.429385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:12.429386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:12.429395 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:12.430575 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:12.430967 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:12.430968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:12.430969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:12.431031 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:12.431292 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:12.431293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:12.431293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:12.431411 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:12.431490 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:12.431760 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 12:59:12.431858 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:59:12.432157 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.423351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:56081)
0806 12:59:12.423546 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:12.423585 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:12.424154 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:12.424361 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:12.424365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:12.425058 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:12.425263 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.425274 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.426022 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.426023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.426885 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.426888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.427321 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.427323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.427451 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.427925 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:12.427936 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:12.429287 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:12.430660 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:12.430667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:12.430669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:12.430891 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:12.431101 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:12.431101 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:12.431102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:12.431208 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:12.431480 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:12.431487 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:12.431626 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 12:59:12.431736 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 12:59:12.431895 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260806 12:59:12.432665 6999 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33939
I20260806 12:59:12.432746 6999 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:12.432889 6999 heartbeater.cc:511] Master 127.24.83.62:33939 requested a full tablet report, sending...
I20260806 12:59:12.433220 6534 ts_manager.cc:194] Registered new tserver with Master: b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745)
I20260806 12:59:12.433549 6534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:56081
I20260806 12:59:12.436573 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:59:12.443295 6585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.437860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51550 (local address 127.24.83.62:33939)
0806 12:59:12.437900 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:12.437902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:12.438001 (+ 99us) server_negotiation.cc:407] Connection header received
0806 12:59:12.438053 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:12.438054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:12.438062 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:12.438108 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:12.438614 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.439064 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.439735 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.439980 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.440031 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.441160 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:12.441160 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:12.441161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:12.441165 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:12.442350 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:12.442688 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:12.442689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:12.442689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:12.442736 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:12.442898 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:12.442898 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:12.442898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:12.443025 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:12.443102 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:12.443182 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 12:59:12.443236 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 12:59:12.445021 6534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:59:12.446587 6534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:59:12.453114 7010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.448803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:35745 (local address 127.0.0.1:53394)
0806 12:59:12.448944 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:12.448974 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:12.449054 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:12.449421 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:12.449424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:12.449425 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:12.449494 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.449496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.450637 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.450638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.451855 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.451858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.452763 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.452764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.452856 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.452870 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 12:59:12.452976 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 12:59:12.453034 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260806 12:59:12.453130 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.448924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53394 (local address 127.24.83.3:35745)
0806 12:59:12.449212 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:12.449217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:12.449227 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:59:12.449266 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:12.449282 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:12.449301 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:12.449388 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:12.449577 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.450543 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.451961 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.452666 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.452760 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.452820 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:12.452993 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 12:59:12.453033 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":84,"threads_started":1}
I20260806 12:59:12.453573 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.449188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:44351 (local address 127.0.0.1:57644)
0806 12:59:12.449869 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:12.449880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:12.449934 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:12.450102 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:12.450103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:12.450105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:12.450144 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.450145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.451346 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.451347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.452440 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.452443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.453331 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.453332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.453404 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.453408 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:12.453464 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 12:59:12.453507 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":1,"thread_start_us":69,"threads_started":1}
I20260806 12:59:12.453619 7015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.449312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57644 (local address 127.24.83.1:44351)
0806 12:59:12.449529 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:12.449534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:12.449939 (+ 405us) server_negotiation.cc:407] Connection header received
0806 12:59:12.450001 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:12.450004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:12.450019 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:12.450093 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:12.450205 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.451246 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.452530 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.453240 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.453318 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.453368 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:12.453489 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 12:59:12.453536 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":91,"threads_started":1}
I20260806 12:59:12.454021 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.449160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48730 (local address 127.24.83.2:44315)
0806 12:59:12.449617 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:12.449621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:12.449630 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:59:12.449668 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:12.449670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:12.449686 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:12.449766 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:12.449916 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.451018 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.452280 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.452923 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.452992 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.453065 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:12.453896 (+ 831us) server_negotiation.cc:299] Negotiation successful
0806 12:59:12.453945 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":91,"threads_started":1}
I20260806 12:59:12.453946 6932 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0187176be044da84947883b9488221 (DEFAULT_TABLE table=test-table [id=424c2abd6c184be9aed1a2999081dbff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:12.454216 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.449023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:44315 (local address 127.0.0.1:48730)
0806 12:59:12.449342 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:12.449377 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:12.449425 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:12.449770 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:12.449771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:12.449773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:12.449817 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.449818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.451115 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.451116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.452177 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.452179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.453103 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.453104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.453489 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.453496 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 12:59:12.453549 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 12:59:12.453590 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":65,"threads_started":1}
I20260806 12:59:12.454342 6932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a0187176be044da84947883b9488221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:12.454252 6661 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0187176be044da84947883b9488221 (DEFAULT_TABLE table=test-table [id=424c2abd6c184be9aed1a2999081dbff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:12.454578 6661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a0187176be044da84947883b9488221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:12.454905 6797 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0187176be044da84947883b9488221 (DEFAULT_TABLE table=test-table [id=424c2abd6c184be9aed1a2999081dbff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:12.455227 6797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a0187176be044da84947883b9488221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:12.457612 7016 tablet_bootstrap.cc:492] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Bootstrap starting.
I20260806 12:59:12.457953 7017 tablet_bootstrap.cc:492] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: Bootstrap starting.
I20260806 12:59:12.458387 7016 tablet_bootstrap.cc:654] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:12.458557 7018 tablet_bootstrap.cc:492] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: Bootstrap starting.
I20260806 12:59:12.458765 7017 tablet_bootstrap.cc:654] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:12.459108 7016 log.cc:824] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:12.459164 7018 tablet_bootstrap.cc:654] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:12.459465 7017 log.cc:824] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:12.459779 7018 log.cc:824] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:12.460063 7016 tablet_bootstrap.cc:492] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: No bootstrap required, opened a new log
I20260806 12:59:12.460132 7016 ts_tablet_manager.cc:1397] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 12:59:12.460275 7017 tablet_bootstrap.cc:492] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: No bootstrap required, opened a new log
I20260806 12:59:12.460347 7017 ts_tablet_manager.cc:1397] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:59:12.460497 7018 tablet_bootstrap.cc:492] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: No bootstrap required, opened a new log
I20260806 12:59:12.460561 7018 ts_tablet_manager.cc:1397] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:59:12.461611 7016 raft_consensus.cc:348] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:12.461766 7016 raft_consensus.cc:374] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:12.461812 7016 raft_consensus.cc:729] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc507127e3f64fd19e6c814055a0e35b, State: Initialized, Role: FOLLOWER
I20260806 12:59:12.461962 7016 consensus_queue.cc:260] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:12.462225 7016 ts_tablet_manager.cc:1428] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:59:12.462376 6728 heartbeater.cc:503] Master 127.24.83.62:33939 was elected leader, sending a full tablet report...
I20260806 12:59:12.462301 7017 raft_consensus.cc:348] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:12.462474 7017 raft_consensus.cc:374] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:12.462519 7017 raft_consensus.cc:729] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66d672365434791a0d2d43d60ce91db, State: Initialized, Role: FOLLOWER
I20260806 12:59:12.462459 7018 raft_consensus.cc:348] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:12.462602 7018 raft_consensus.cc:374] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:12.462647 7018 raft_consensus.cc:729] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24eebe04e534ca6b37caf56fae9bbd7, State: Initialized, Role: FOLLOWER
I20260806 12:59:12.462672 7017 consensus_queue.cc:260] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:12.462738 7018 consensus_queue.cc:260] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:12.462954 7017 ts_tablet_manager.cc:1428] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 12:59:12.463006 7018 ts_tablet_manager.cc:1428] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 12:59:12.463116 6999 heartbeater.cc:503] Master 127.24.83.62:33939 was elected leader, sending a full tablet report...
I20260806 12:59:12.463156 6864 heartbeater.cc:503] Master 127.24.83.62:33939 was elected leader, sending a full tablet report...
W20260806 12:59:12.500448 6865 tablet.cc:2369] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:59:12.572887 6729 tablet.cc:2369] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:59:12.673247 7000 tablet.cc:2369] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:59:12.678723 7022 raft_consensus.cc:482] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:59:12.678838 7022 raft_consensus.cc:504] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:12.679275 7022 leader_election.cc:288] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745), d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315)
I20260806 12:59:12.682315 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.679556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:49093 (local address 127.24.83.3:35745)
0806 12:59:12.679624 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:12.679627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:12.679788 (+ 161us) server_negotiation.cc:407] Connection header received
0806 12:59:12.679819 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:12.679820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:12.679831 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:12.679878 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:12.680100 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.680602 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.681552 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.681943 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.682029 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.682040 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:12.682198 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 12:59:12.682234 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 12:59:12.682340 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.679750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:35859 (local address 127.24.83.2:44315)
0806 12:59:12.679803 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:12.679805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:12.679815 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:59:12.679907 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:12.679908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:12.679920 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:12.680014 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:12.680199 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.680675 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.681592 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.681979 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.682060 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.682070 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:12.682237 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 12:59:12.682272 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 12:59:12.682350 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.679457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:35745 (local address 127.24.83.1:49093)
0806 12:59:12.679619 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:12.679633 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:12.679728 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:12.679913 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:12.679914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:12.679951 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:12.680027 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.680029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.680687 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.680688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.681460 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.681462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.682030 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.682032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.682109 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.682113 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:12.682192 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 12:59:12.682233 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260806 12:59:12.682361 7026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.679665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:44315 (local address 127.24.83.1:35859)
0806 12:59:12.679762 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:12.679772 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:12.679830 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:12.680031 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:12.680031 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:12.680034 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:12.680120 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.680121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.680763 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.680764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.681501 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:12.681502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.682061 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.682062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:12.682143 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.682147 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:12.682229 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 12:59:12.682268 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":58,"threads_started":1}
I20260806 12:59:12.682813 6953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0187176be044da84947883b9488221" candidate_uuid: "dc507127e3f64fd19e6c814055a0e35b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66d672365434791a0d2d43d60ce91db" is_pre_election: true
I20260806 12:59:12.682823 6817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0187176be044da84947883b9488221" candidate_uuid: "dc507127e3f64fd19e6c814055a0e35b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" is_pre_election: true
I20260806 12:59:12.683001 6953 raft_consensus.cc:2454] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc507127e3f64fd19e6c814055a0e35b in term 0.
I20260806 12:59:12.683001 6817 raft_consensus.cc:2454] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc507127e3f64fd19e6c814055a0e35b in term 0.
I20260806 12:59:12.683218 6614 leader_election.cc:302] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d24eebe04e534ca6b37caf56fae9bbd7, dc507127e3f64fd19e6c814055a0e35b; no voters:
I20260806 12:59:12.683338 7022 raft_consensus.cc:2781] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:59:12.683399 7022 raft_consensus.cc:482] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:59:12.683422 7022 raft_consensus.cc:3037] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:12.684248 7022 raft_consensus.cc:504] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:12.684376 7022 leader_election.cc:288] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 1 election: Requested vote from peers b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745), d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315)
I20260806 12:59:12.684541 6953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0187176be044da84947883b9488221" candidate_uuid: "dc507127e3f64fd19e6c814055a0e35b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66d672365434791a0d2d43d60ce91db"
I20260806 12:59:12.684563 6817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0187176be044da84947883b9488221" candidate_uuid: "dc507127e3f64fd19e6c814055a0e35b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d24eebe04e534ca6b37caf56fae9bbd7"
I20260806 12:59:12.684623 6953 raft_consensus.cc:3037] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:12.684659 6817 raft_consensus.cc:3037] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:12.685334 6953 raft_consensus.cc:2454] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc507127e3f64fd19e6c814055a0e35b in term 1.
I20260806 12:59:12.685474 6817 raft_consensus.cc:2454] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc507127e3f64fd19e6c814055a0e35b in term 1.
I20260806 12:59:12.685510 6614 leader_election.cc:302] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b66d672365434791a0d2d43d60ce91db, dc507127e3f64fd19e6c814055a0e35b; no voters:
I20260806 12:59:12.685590 7022 raft_consensus.cc:2781] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:59:12.685766 7022 raft_consensus.cc:686] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 1 LEADER]: Becoming Leader. State: Replica: dc507127e3f64fd19e6c814055a0e35b, State: Running, Role: LEADER
I20260806 12:59:12.685900 7022 consensus_queue.cc:237] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:12.686774 6533 catalog_manager.cc:5495] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b reported cstate change: term changed from 0 to 1, leader changed from <none> to dc507127e3f64fd19e6c814055a0e35b (127.24.83.1). New cstate: current_term: 1 leader_uuid: "dc507127e3f64fd19e6c814055a0e35b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } health_report { overall_health: HEALTHY } } }
I20260806 12:59:12.733501 7015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.731258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.24.83.1:44351)
0806 12:59:12.731317 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:12.731323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:12.731332 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:59:12.731371 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:12.731372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:12.731391 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:12.731454 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:59:12.731576 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.732114 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.732846 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:12.733033 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:12.733086 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:12.733223 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:59:12.733327 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:59:12.733390 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:12.733426 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 12:59:12.733441 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 12:59:12.735405 6953 raft_consensus.cc:1261] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db [term 1 FOLLOWER]: Refusing update from remote peer dc507127e3f64fd19e6c814055a0e35b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:12.735431 6817 raft_consensus.cc:1261] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7 [term 1 FOLLOWER]: Refusing update from remote peer dc507127e3f64fd19e6c814055a0e35b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:12.735793 7022 consensus_queue.cc:1035] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:12.735970 7027 consensus_queue.cc:1035] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:12.739024 7032 mvcc.cc:205] Tried to move back new op lower bound from 7315542641601425408 to 7315542641402179584. Current Snapshot: MvccSnapshot[applied={T|T < 7315542641601425408}]
I20260806 12:59:12.740797 6534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:59:12.740911 6534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:59:12.741659 6534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Sending DeleteTablet for 3 replicas of tablet 7a0187176be044da84947883b9488221
I20260806 12:59:12.741968 6932 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a0187176be044da84947883b9488221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:12 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:53394
I20260806 12:59:12.742048 6797 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a0187176be044da84947883b9488221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:12 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:48730
I20260806 12:59:12.742074 6661 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a0187176be044da84947883b9488221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:12 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:57644
I20260806 12:59:12.742189 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 6597
I20260806 12:59:12.743216 7040 tablet_replica.cc:330] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: stopping tablet replica
I20260806 12:59:12.743346 7040 raft_consensus.cc:2229] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:59:12.743455 7040 raft_consensus.cc:2258] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:12.743944 7040 ts_tablet_manager.cc:1905] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:59:12.744235 7039 tablet_replica.cc:330] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: stopping tablet replica
I20260806 12:59:12.744323 7039 raft_consensus.cc:2229] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:59:12.744431 7039 raft_consensus.cc:2258] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:12.745085 7039 ts_tablet_manager.cc:1905] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:59:12.745829 7040 ts_tablet_manager.cc:1918] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:12.745926 7040 log.cc:1196] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/7a0187176be044da84947883b9488221
I20260806 12:59:12.746169 7040 ts_tablet_manager.cc:1939] T 7a0187176be044da84947883b9488221 P d24eebe04e534ca6b37caf56fae9bbd7: Deleting consensus metadata
I20260806 12:59:12.746577 6519 catalog_manager.cc:4841] TS d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315): tablet 7a0187176be044da84947883b9488221 (table test-table [id=424c2abd6c184be9aed1a2999081dbff]) successfully deleted
I20260806 12:59:12.746973 7039 ts_tablet_manager.cc:1918] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:12.747052 7039 log.cc:1196] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/7a0187176be044da84947883b9488221
I20260806 12:59:12.747328 7039 ts_tablet_manager.cc:1939] T 7a0187176be044da84947883b9488221 P b66d672365434791a0d2d43d60ce91db: Deleting consensus metadata
I20260806 12:59:12.747820 6519 catalog_manager.cc:4841] TS b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745): tablet 7a0187176be044da84947883b9488221 (table test-table [id=424c2abd6c184be9aed1a2999081dbff]) successfully deleted
W20260806 12:59:12.749495 6519 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:44351 (error 108)
I20260806 12:59:12.749626 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 6733
I20260806 12:59:12.750422 7010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:12.750318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:44351 (local address 127.0.0.1:57664)
0806 12:59:12.750359 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:12.750395 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 12:59:12.750545 6519 catalog_manager.cc:4584] TS dc507127e3f64fd19e6c814055a0e35b (127.24.83.1:44351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a0187176be044da84947883b9488221: Network error: Client connection negotiation failed: client connection to 127.24.83.1:44351: connect: Connection refused (error 111)
I20260806 12:59:12.755643 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 6869
I20260806 12:59:12.761234 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 6502
I20260806 12:59:12.767045 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33939
--webserver_interface=127.24.83.62
--webserver_port=36633
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:33939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:12.847841 7043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:12.848050 7043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:12.848071 7043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:12.849143 7043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:59:12.849174 7043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:12.849188 7043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:12.849201 7043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:59:12.849213 7043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:59:12.850585 7043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:33939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=36633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:12.850778 7043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:12.850940 7043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:12.860502 7050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:12.860532 7043 server_base.cc:1034] running on GCE node
W20260806 12:59:12.860502 7052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:12.860531 7049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:12.861032 7043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:12.861336 7043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:12.862500 7043 hybrid_clock.cc:648] HybridClock initialized: now 1786021152862487 us; error 34 us; skew 500 ppm
Aug 06 12:59:12 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:12.865136 7043 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:59:12.865586 7043 webserver.cc:460] Webserver started at http://127.24.83.62:36633/ using document root <none> and password file <none>
I20260806 12:59:12.865845 7043 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:12.865900 7043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:12.867300 7043 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:12.867990 7059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:12.868142 7043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:59:12.868230 7043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "478b4505ddc5406595f298298d9ce226"
format_stamp: "Formatted at 2026-08-06 12:59:11 on dist-test-slave-zkpd"
server_key: "2a8393524e63944bde2e66468bad61b6"
server_key_iv: "5af0b83a053c3056405ba267afd323f0"
server_key_version: "encryptionkey@0"
I20260806 12:59:12.868608 7043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:12.889096 7043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:12.891471 7043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:12.891582 7043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:12.895606 7043 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:33939
I20260806 12:59:12.895699 7113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:33939 every 8 connection(s)
I20260806 12:59:12.896097 7043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:59:12.897787 7114 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:59:12.898393 7114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Bootstrap starting.
I20260806 12:59:12.902433 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 7043
I20260806 12:59:12.902796 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:44351
--local_ip_for_outbound_sockets=127.24.83.1
--tserver_master_addrs=127.24.83.62:33939
--webserver_port=44733
--webserver_interface=127.24.83.1
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:12.908615 7114 log.cc:824] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:12.910653 7114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:59:12.910986 7114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Bootstrap complete.
I20260806 12:59:12.913380 7114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:12.913609 7114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478b4505ddc5406595f298298d9ce226, State: Initialized, Role: FOLLOWER
I20260806 12:59:12.913800 7114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:12.913861 7114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:59:12.913914 7114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:59:12.913961 7114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:59:12.914948 7114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:12.915063 7114 leader_election.cc:302] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 478b4505ddc5406595f298298d9ce226; no voters:
I20260806 12:59:12.915263 7114 leader_election.cc:288] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 12:59:12.915302 7118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 12:59:12.915437 7118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 2 LEADER]: Becoming Leader. State: Replica: 478b4505ddc5406595f298298d9ce226, State: Running, Role: LEADER
I20260806 12:59:12.915530 7114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:59:12.915524 7118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:12.915935 7119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 478b4505ddc5406595f298298d9ce226. Latest consensus state: current_term: 2 leader_uuid: "478b4505ddc5406595f298298d9ce226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } } }
I20260806 12:59:12.916015 7119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:12.916242 7122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:59:12.917029 7122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:59:12.917124 7122 catalog_manager.cc:1259] Loaded cluster ID: 7559899162e9487f8050f0a3ac061f6d
I20260806 12:59:12.917150 7122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:59:12.917459 7120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "478b4505ddc5406595f298298d9ce226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } } }
I20260806 12:59:12.917495 7122 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 12:59:12.917609 7120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:12.922240 7122 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 12:59:12.924710 7122 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:59:12.924938 7122 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Loaded TSK: 0
I20260806 12:59:12.925143 7122 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 12:59:12.989162 7116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:12.989365 7116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:12.989389 7116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:12.990564 7116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:12.990609 7116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:12.990635 7116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:59:12.992111 7116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:44351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=44733
--tserver_master_addrs=127.24.83.62:33939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:12.992348 7116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:12.992595 7116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:12.997483 7141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:12.997608 7142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:12.997479 7144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:12.998104 7116 server_base.cc:1034] running on GCE node
I20260806 12:59:12.998313 7116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:12.998579 7116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:12.999778 7116 hybrid_clock.cc:648] HybridClock initialized: now 1786021152999752 us; error 40 us; skew 500 ppm
Aug 06 12:59:12 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:13.002482 7116 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:59:13.002995 7116 webserver.cc:460] Webserver started at http://127.24.83.1:44733/ using document root <none> and password file <none>
I20260806 12:59:13.003223 7116 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:13.003309 7116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:13.004760 7116 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:59:13.005517 7151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.005712 7116 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:59:13.005821 7116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "dc507127e3f64fd19e6c814055a0e35b"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "f8a98ec0b1c820f9ebd839bd5dcc33b0"
server_key_iv: "34945ae537ac38b298de40f0d9d35964"
server_key_version: "encryptionkey@0"
I20260806 12:59:13.006213 7116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:13.019713 7116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:13.022063 7116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:13.022228 7116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:13.022496 7116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:13.023012 7158 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 12:59:13.024616 7116 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 12:59:13.024667 7116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 12:59:13.024699 7116 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 12:59:13.025354 7116 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 12:59:13.025399 7116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 12:59:13.025449 7158 tablet_bootstrap.cc:492] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Bootstrap starting.
I20260806 12:59:13.033486 7116 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:44351
I20260806 12:59:13.034003 7116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:59:13.034807 7265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:44351 every 8 connection(s)
I20260806 12:59:13.037783 7158 log.cc:824] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:13.039085 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 7116
I20260806 12:59:13.039428 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:44315
--local_ip_for_outbound_sockets=127.24.83.2
--tserver_master_addrs=127.24.83.62:33939
--webserver_port=38135
--webserver_interface=127.24.83.2
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:13.040766 7158 tablet_bootstrap.cc:492] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:59:13.041070 7158 tablet_bootstrap.cc:492] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Bootstrap complete.
I20260806 12:59:13.042366 7158 ts_tablet_manager.cc:1397] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260806 12:59:13.043859 7158 raft_consensus.cc:348] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.044265 7158 raft_consensus.cc:729] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc507127e3f64fd19e6c814055a0e35b, State: Initialized, Role: FOLLOWER
Aug 06 12:59:13 dist-test-slave-zkpd krb5kdc[6488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:13.044442 7158 consensus_queue.cc:260] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.044723 7158 ts_tablet_manager.cc:1428] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260806 12:59:13.049299 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.034840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:34851)
0806 12:59:13.036161 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.036207 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:13.036803 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:13.037376 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:13.037380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:13.037755 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:13.038115 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.038122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.040088 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.040089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.041361 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.041364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.042272 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.042273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.042420 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.043045 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:13.043055 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:13.044857 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:13.047386 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:13.047394 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:13.047397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:13.047676 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:13.048013 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:13.048014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:13.048015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:13.048143 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:13.048650 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:13.048656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:13.048804 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 12:59:13.048928 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 12:59:13.049093 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1273,"thread_start_us":106,"threads_started":1}
I20260806 12:59:13.049438 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.034913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:34851 (local address 127.24.83.62:33939)
0806 12:59:13.036044 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:13.036056 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:13.036362 (+ 306us) server_negotiation.cc:407] Connection header received
0806 12:59:13.037063 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:13.037071 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:13.037149 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:13.037337 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:13.038324 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.039950 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.041530 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.042169 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.042748 (+ 579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.044980 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:13.044985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:13.044985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:13.044997 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:13.047274 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:13.047772 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:13.047776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:13.047777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:13.047936 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:13.048222 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:13.048223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:13.048223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:13.048563 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:13.048660 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:13.049012 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 12:59:13.049167 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1071,"thread_start_us":76,"threads_started":1}
I20260806 12:59:13.050041 7266 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33939
I20260806 12:59:13.050132 7266 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:13.050314 7266 heartbeater.cc:511] Master 127.24.83.62:33939 requested a full tablet report, sending...
I20260806 12:59:13.051000 7078 ts_manager.cc:194] Registered new tserver with Master: dc507127e3f64fd19e6c814055a0e35b (127.24.83.1:44351)
I20260806 12:59:13.051959 7078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:34851
I20260806 12:59:13.052784 7266 heartbeater.cc:503] Master 127.24.83.62:33939 was elected leader, sending a full tablet report...
Aug 06 12:59:13 dist-test-slave-zkpd krb5kdc[6488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:59:13.059995 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.051553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:44351 (local address 127.0.0.1:57666)
0806 12:59:13.051748 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.051783 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:13.051846 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:13.052924 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:13.052931 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:13.053245 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:13.053297 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.053299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.054115 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.054116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.054678 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.054680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.055196 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.055198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.055272 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.055751 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:13.055756 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:13.057098 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:13.058832 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:13.058840 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:13.058842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:13.059036 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:13.059249 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:13.059250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:13.059251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:13.059308 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:13.059598 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:13.059605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:13.059652 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 12:59:13.059744 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 12:59:13.059860 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":97,"threads_started":1}
I20260806 12:59:13.060012 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.051826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57666 (local address 127.24.83.1:44351)
0806 12:59:13.052251 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:13.052255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:13.052266 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:59:13.052298 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:13.052300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:13.052314 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:13.052406 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:13.053375 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.054026 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.054777 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.055085 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.055192 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.057194 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:13.057201 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:13.057202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:13.057208 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:13.058729 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:13.059119 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:13.059122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:13.059122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:13.059180 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:13.059380 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:13.059382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:13.059382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:13.059513 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:13.059618 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:13.059795 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 12:59:13.059878 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":105,"threads_started":1}
I20260806 12:59:13.060369 7194 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a0187176be044da84947883b9488221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:12 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:57666
I20260806 12:59:13.060843 7275 tablet_replica.cc:330] stopping tablet replica
I20260806 12:59:13.060961 7275 raft_consensus.cc:2229] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:59:13.061028 7275 raft_consensus.cc:2258] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:13.061316 7275 ts_tablet_manager.cc:1905] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:59:13.063632 7275 ts_tablet_manager.cc:1918] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:13.063709 7275 log.cc:1196] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal/wals/7a0187176be044da84947883b9488221
I20260806 12:59:13.063961 7275 ts_tablet_manager.cc:1939] T 7a0187176be044da84947883b9488221 P dc507127e3f64fd19e6c814055a0e35b: Deleting consensus metadata
I20260806 12:59:13.064473 7061 catalog_manager.cc:4841] TS dc507127e3f64fd19e6c814055a0e35b (127.24.83.1:44351): tablet 7a0187176be044da84947883b9488221 (table test-table [id=424c2abd6c184be9aed1a2999081dbff]) successfully deleted
W20260806 12:59:13.128962 7271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:13.129169 7271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:13.129194 7271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:13.130350 7271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:13.130393 7271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:13.130419 7271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:59:13.131907 7271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:44315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=38135
--tserver_master_addrs=127.24.83.62:33939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:13.132125 7271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:13.132347 7271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:13.137393 7283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:13.137401 7280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:13.137414 7281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:13.137504 7271 server_base.cc:1034] running on GCE node
I20260806 12:59:13.137846 7271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:13.138109 7271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:13.139274 7271 hybrid_clock.cc:648] HybridClock initialized: now 1786021153139266 us; error 38 us; skew 500 ppm
Aug 06 12:59:13 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:13.141675 7271 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:59:13.142093 7271 webserver.cc:460] Webserver started at http://127.24.83.2:38135/ using document root <none> and password file <none>
I20260806 12:59:13.142330 7271 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:13.142414 7271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:13.143755 7271 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:59:13.144407 7290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.144582 7271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.144681 7271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "d24eebe04e534ca6b37caf56fae9bbd7"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "cf14a3fd2f663b64adc2057bc3ddb81c"
server_key_iv: "f898fdfbf02c4d59c6c86c24e9ec2a66"
server_key_version: "encryptionkey@0"
I20260806 12:59:13.145038 7271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:13.159713 7271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:13.161650 7271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:13.161820 7271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:13.162071 7271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:13.162395 7271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:13.162449 7271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.162490 7271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:13.162535 7271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.168606 7271 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:44315
I20260806 12:59:13.168661 7405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:44315 every 8 connection(s)
I20260806 12:59:13.169008 7271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:59:13 dist-test-slave-zkpd krb5kdc[6488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:13.178082 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 7271
I20260806 12:59:13.178364 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:35745
--local_ip_for_outbound_sockets=127.24.83.3
--tserver_master_addrs=127.24.83.62:33939
--webserver_port=42625
--webserver_interface=127.24.83.3
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:13.178620 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.169711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:34109 (local address 127.24.83.62:33939)
0806 12:59:13.169784 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:13.169786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:13.170010 (+ 224us) server_negotiation.cc:407] Connection header received
0806 12:59:13.170557 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:13.170560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:13.170573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:13.170625 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:13.171634 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.172208 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.173245 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.173462 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.173542 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.175655 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:13.175657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:13.175657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:13.175665 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:13.177127 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:13.177549 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:13.177550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:13.177551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:13.177614 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:13.177895 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:13.177896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:13.177896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:13.178052 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:13.178134 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:13.178430 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 12:59:13.178528 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:59:13.178799 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.169679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:34109)
0806 12:59:13.169882 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.169925 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:13.170442 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:13.170700 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:13.170704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:13.171336 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:13.171530 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.171535 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.172307 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.172308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.173149 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.173151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.173550 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.173552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.173692 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.174226 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:13.174235 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:13.175555 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:13.177229 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:13.177238 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:13.177240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:13.177468 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:13.177681 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:13.177682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:13.177683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:13.177830 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:13.178126 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:13.178134 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:13.178293 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 12:59:13.178406 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 12:59:13.178565 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":111,"threads_started":1}
I20260806 12:59:13.179323 7406 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33939
I20260806 12:59:13.179450 7406 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:13.179633 7406 heartbeater.cc:511] Master 127.24.83.62:33939 requested a full tablet report, sending...
I20260806 12:59:13.180037 7078 ts_manager.cc:194] Registered new tserver with Master: d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315)
I20260806 12:59:13.180435 7078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:34109
W20260806 12:59:13.261401 7409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:13.261605 7409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:13.261628 7409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:13.262864 7409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:13.262909 7409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:13.262965 7409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:59:13.264530 7409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:35745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=42625
--tserver_master_addrs=127.24.83.62:33939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:13.264779 7409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:13.265019 7409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:13.270009 7414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:13.270037 7409 server_base.cc:1034] running on GCE node
W20260806 12:59:13.270063 7415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:13.270200 7417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:13.270411 7409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:13.270695 7409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:13.271860 7409 hybrid_clock.cc:648] HybridClock initialized: now 1786021153271843 us; error 31 us; skew 500 ppm
Aug 06 12:59:13 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:13.274358 7409 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:59:13.274835 7409 webserver.cc:460] Webserver started at http://127.24.83.3:42625/ using document root <none> and password file <none>
I20260806 12:59:13.275077 7409 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:13.275159 7409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:13.276582 7409 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:59:13.277189 7424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.277385 7409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:59:13.277508 7409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "b66d672365434791a0d2d43d60ce91db"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "53bf9b5073132ffbe29a16c2c0c84c56"
server_key_iv: "b26639998f0b4728fd30ad32df02b9df"
server_key_version: "encryptionkey@0"
I20260806 12:59:13.277875 7409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:13.293078 7409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:13.294987 7409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:13.295162 7409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:13.295416 7409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:13.295733 7409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:13.295795 7409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.295840 7409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:13.295871 7409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.301959 7409 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:35745
I20260806 12:59:13.302007 7539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:35745 every 8 connection(s)
I20260806 12:59:13.302348 7409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:59:13.304126 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 7409
Aug 06 12:59:13 dist-test-slave-zkpd krb5kdc[6488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:13.312191 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.304561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51556 (local address 127.24.83.62:33939)
0806 12:59:13.304690 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:13.304692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:13.304702 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:59:13.304821 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:13.304821 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:13.304831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:13.304889 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:13.305557 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.306291 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.307373 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.307569 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.307654 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.309160 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:13.309161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:13.309162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:13.309170 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:13.310951 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:13.311307 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:13.311308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:13.311309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:13.311376 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:13.311580 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:13.311581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:13.311582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:13.311708 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:13.311930 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:13.311970 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 12:59:13.312052 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260806 12:59:13.313246 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.303183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:48157 (local address 127.24.83.62:33939)
0806 12:59:13.303276 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:13.303278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:13.303389 (+ 111us) server_negotiation.cc:407] Connection header received
0806 12:59:13.303990 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:13.303991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:13.304002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:13.304049 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:13.305473 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.306283 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.307638 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.307850 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.307967 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.310121 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:13.310122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:13.310123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:13.310130 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:13.311548 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:13.312000 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:13.312001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:13.312001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:13.312062 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:13.312373 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:13.312374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:13.312375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:13.312509 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:13.312574 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:13.313088 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 12:59:13.313175 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 12:59:13.313513 7542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.303097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:48157)
0806 12:59:13.303276 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.303317 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:13.303837 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:13.304130 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:13.304136 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:13.305054 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:13.305377 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.305384 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.306390 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.306391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.307562 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.307565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.307984 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.307985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.308158 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.308645 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:13.308653 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:13.310050 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:13.311656 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:13.311663 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:13.311665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:13.311937 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:13.312131 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:13.312132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:13.312133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:13.312292 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:13.312608 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:13.312617 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:13.312883 (+ 266us) client_negotiation.cc:769] Sending connection context
0806 12:59:13.313003 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 12:59:13.313233 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260806 12:59:13.314007 7540 heartbeater.cc:347] Connected to a master server at 127.24.83.62:33939
I20260806 12:59:13.314116 7540 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:13.314298 7540 heartbeater.cc:511] Master 127.24.83.62:33939 requested a full tablet report, sending...
I20260806 12:59:13.314678 7078 ts_manager.cc:194] Registered new tserver with Master: b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745)
I20260806 12:59:13.315014 7078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:48157
I20260806 12:59:13.324139 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:59:13.327167 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.324515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51564 (local address 127.24.83.62:33939)
0806 12:59:13.324572 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:13.324574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:13.324686 (+ 112us) server_negotiation.cc:407] Connection header received
0806 12:59:13.324815 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:13.324816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:13.324863 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:13.324953 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:59:13.325090 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.325578 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.326363 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.326566 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.326652 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.326837 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:59:13.326947 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:59:13.327045 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:13.327064 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 12:59:13.327082 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 12:59:13.327509 7078 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:59:13.329334 7078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:59:13.332098 7194 tablet_service.cc:1467] Processing CreateTablet for tablet 0e16cfe5c67d4290ab0971348613b928 (DEFAULT_TABLE table=test-table [id=32c4d5e694f046a79f98742382a49761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:13.332317 7194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e16cfe5c67d4290ab0971348613b928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:13.333606 7158 tablet_bootstrap.cc:492] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: Bootstrap starting.
I20260806 12:59:13.333946 7158 tablet_bootstrap.cc:654] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:13.334687 7158 tablet_bootstrap.cc:492] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: No bootstrap required, opened a new log
I20260806 12:59:13.334754 7158 ts_tablet_manager.cc:1397] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:13.335088 7158 raft_consensus.cc:348] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.335175 7158 raft_consensus.cc:374] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:13.335227 7158 raft_consensus.cc:729] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc507127e3f64fd19e6c814055a0e35b, State: Initialized, Role: FOLLOWER
I20260806 12:59:13.335273 7158 consensus_queue.cc:260] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.335371 7158 ts_tablet_manager.cc:1428] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:13.335758 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.331524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:35745 (local address 127.0.0.1:53402)
0806 12:59:13.331594 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.331605 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:13.331646 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:13.332048 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:13.332049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:13.332051 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:13.332099 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.332100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.333394 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.333395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.334536 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.334539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.335500 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.335502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.335575 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.335598 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 12:59:13.335651 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 12:59:13.335695 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 12:59:13.335927 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.331653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:44315 (local address 127.0.0.1:48738)
0806 12:59:13.331861 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.331873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:13.331944 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:13.332247 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:13.332248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:13.332250 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:13.332299 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.332301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.333574 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.333575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.334691 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.334694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.335593 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.335594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.335691 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.335695 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:13.335784 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 12:59:13.335825 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":100,"threads_started":1}
I20260806 12:59:13.335947 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.331767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48738 (local address 127.24.83.2:44315)
0806 12:59:13.332079 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:13.332084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:13.332094 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:59:13.332129 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:13.332132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:13.332149 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:13.332374 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:13.332451 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.333473 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.334795 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.335451 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.335506 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.335558 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:13.335806 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 12:59:13.335846 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":93,"threads_started":1}
I20260806 12:59:13.336575 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.331619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53402 (local address 127.24.83.3:35745)
0806 12:59:13.331841 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:13.331847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:13.331859 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:59:13.331926 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:13.331929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:13.331947 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:13.332025 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:13.332159 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.333274 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.334631 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.335379 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.335493 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.335558 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:13.336461 (+ 903us) server_negotiation.cc:299] Negotiation successful
0806 12:59:13.336499 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":130,"threads_started":1}
I20260806 12:59:13.336951 7338 tablet_service.cc:1467] Processing CreateTablet for tablet 0e16cfe5c67d4290ab0971348613b928 (DEFAULT_TABLE table=test-table [id=32c4d5e694f046a79f98742382a49761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:13.337342 7338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e16cfe5c67d4290ab0971348613b928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:13.337239 7473 tablet_service.cc:1467] Processing CreateTablet for tablet 0e16cfe5c67d4290ab0971348613b928 (DEFAULT_TABLE table=test-table [id=32c4d5e694f046a79f98742382a49761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:13.337592 7473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e16cfe5c67d4290ab0971348613b928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:13.341068 7551 tablet_bootstrap.cc:492] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db: Bootstrap starting.
I20260806 12:59:13.341809 7551 tablet_bootstrap.cc:654] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:13.342324 7551 log.cc:824] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:13.342651 7552 tablet_bootstrap.cc:492] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: Bootstrap starting.
I20260806 12:59:13.343066 7551 tablet_bootstrap.cc:492] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db: No bootstrap required, opened a new log
I20260806 12:59:13.343124 7551 ts_tablet_manager.cc:1397] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:59:13.343246 7552 tablet_bootstrap.cc:654] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:13.343703 7552 log.cc:824] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:13.344408 7552 tablet_bootstrap.cc:492] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: No bootstrap required, opened a new log
I20260806 12:59:13.344486 7552 ts_tablet_manager.cc:1397] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 12:59:13.344739 7551 raft_consensus.cc:348] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.344923 7551 raft_consensus.cc:374] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:13.344976 7551 raft_consensus.cc:729] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66d672365434791a0d2d43d60ce91db, State: Initialized, Role: FOLLOWER
I20260806 12:59:13.345125 7551 consensus_queue.cc:260] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.345415 7551 ts_tablet_manager.cc:1428] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 12:59:13.345466 7540 heartbeater.cc:503] Master 127.24.83.62:33939 was elected leader, sending a full tablet report...
I20260806 12:59:13.346419 7552 raft_consensus.cc:348] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.346542 7552 raft_consensus.cc:374] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:13.346563 7552 raft_consensus.cc:729] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24eebe04e534ca6b37caf56fae9bbd7, State: Initialized, Role: FOLLOWER
I20260806 12:59:13.346691 7552 consensus_queue.cc:260] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.346982 7552 ts_tablet_manager.cc:1428] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 12:59:13.347014 7406 heartbeater.cc:503] Master 127.24.83.62:33939 was elected leader, sending a full tablet report...
W20260806 12:59:13.419570 7407 tablet.cc:2369] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:59:13.462811 7555 raft_consensus.cc:482] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:59:13.462968 7555 raft_consensus.cc:504] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.463419 7555 leader_election.cc:288] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315), dc507127e3f64fd19e6c814055a0e35b (127.24.83.1:44351)
I20260806 12:59:13.467692 7542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.463607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:44315 (local address 127.24.83.3:46911)
0806 12:59:13.463677 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.463692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:13.463769 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:13.464119 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:13.464121 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:13.464122 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:13.464201 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.464202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.465232 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.465233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.466531 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.466534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.467421 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.467423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.467508 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.467511 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:59:13.467575 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 12:59:13.467619 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":59}
I20260806 12:59:13.467691 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.463717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:46911 (local address 127.24.83.2:44315)
0806 12:59:13.463955 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:13.463957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:13.463966 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:59:13.463998 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:13.463999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:13.464013 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:13.464063 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:13.464293 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.465145 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.466634 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.467333 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.467412 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.467423 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:13.467581 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 12:59:13.467619 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212}
I20260806 12:59:13.468019 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.463714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:44351 (local address 127.24.83.3:32977)
0806 12:59:13.463916 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.463926 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:13.463989 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:13.464201 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:13.464202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:13.464203 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:13.464263 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.464264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.465152 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.465153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.466548 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:13.466549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.467468 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.467469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:13.467532 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.467535 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:59:13.467880 (+ 345us) client_negotiation.cc:240] Negotiation successful
0806 12:59:13.467953 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":93,"thread_start_us":97,"threads_started":1}
I20260806 12:59:13.468042 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.463835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:32977 (local address 127.24.83.1:44351)
0806 12:59:13.463881 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:13.463883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:13.463992 (+ 109us) server_negotiation.cc:407] Connection header received
0806 12:59:13.464065 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:13.464066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:13.464095 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:13.464138 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:13.464317 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.465064 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.466676 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.467348 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.467427 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.467523 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:13.467926 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 12:59:13.467974 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 12:59:13.468344 7220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e16cfe5c67d4290ab0971348613b928" candidate_uuid: "b66d672365434791a0d2d43d60ce91db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc507127e3f64fd19e6c814055a0e35b" is_pre_election: true
I20260806 12:59:13.468572 7220 raft_consensus.cc:2454] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b66d672365434791a0d2d43d60ce91db in term 0.
I20260806 12:59:13.468834 7426 leader_election.cc:302] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b66d672365434791a0d2d43d60ce91db, dc507127e3f64fd19e6c814055a0e35b; no voters:
I20260806 12:59:13.468986 7555 raft_consensus.cc:2781] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:59:13.469043 7555 raft_consensus.cc:482] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:59:13.469065 7555 raft_consensus.cc:3037] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:13.469300 7358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e16cfe5c67d4290ab0971348613b928" candidate_uuid: "b66d672365434791a0d2d43d60ce91db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" is_pre_election: true
I20260806 12:59:13.469441 7358 raft_consensus.cc:2454] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b66d672365434791a0d2d43d60ce91db in term 0.
I20260806 12:59:13.469934 7555 raft_consensus.cc:504] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.470044 7555 leader_election.cc:288] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [CANDIDATE]: Term 1 election: Requested vote from peers d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315), dc507127e3f64fd19e6c814055a0e35b (127.24.83.1:44351)
I20260806 12:59:13.470230 7358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e16cfe5c67d4290ab0971348613b928" candidate_uuid: "b66d672365434791a0d2d43d60ce91db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d24eebe04e534ca6b37caf56fae9bbd7"
I20260806 12:59:13.470292 7220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e16cfe5c67d4290ab0971348613b928" candidate_uuid: "b66d672365434791a0d2d43d60ce91db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc507127e3f64fd19e6c814055a0e35b"
I20260806 12:59:13.470314 7358 raft_consensus.cc:3037] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:13.470415 7220 raft_consensus.cc:3037] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:13.471077 7358 raft_consensus.cc:2454] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b66d672365434791a0d2d43d60ce91db in term 1.
I20260806 12:59:13.471227 7220 raft_consensus.cc:2454] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b66d672365434791a0d2d43d60ce91db in term 1.
I20260806 12:59:13.471400 7426 leader_election.cc:302] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b66d672365434791a0d2d43d60ce91db, dc507127e3f64fd19e6c814055a0e35b; no voters:
I20260806 12:59:13.471508 7555 raft_consensus.cc:2781] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:59:13.471659 7555 raft_consensus.cc:686] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 1 LEADER]: Becoming Leader. State: Replica: b66d672365434791a0d2d43d60ce91db, State: Running, Role: LEADER
I20260806 12:59:13.471751 7555 consensus_queue.cc:237] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.472609 7078 catalog_manager.cc:5495] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db reported cstate change: term changed from 0 to 1, leader changed from <none> to b66d672365434791a0d2d43d60ce91db (127.24.83.3). New cstate: current_term: 1 leader_uuid: "b66d672365434791a0d2d43d60ce91db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } health_report { overall_health: UNKNOWN } } }
I20260806 12:59:13.481546 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.479300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.24.83.3:35745)
0806 12:59:13.479355 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:13.479357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:13.479366 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:59:13.479426 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:13.479427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:13.479445 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:13.479497 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:59:13.479598 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.480151 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.480889 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:13.481067 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:13.481140 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:13.481239 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:59:13.481327 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:59:13.481380 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:13.481416 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 12:59:13.481431 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 12:59:13.483256 7220 raft_consensus.cc:1261] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Refusing update from remote peer b66d672365434791a0d2d43d60ce91db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:13.483310 7358 raft_consensus.cc:1261] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7 [term 1 FOLLOWER]: Refusing update from remote peer b66d672365434791a0d2d43d60ce91db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:13.483578 7555 consensus_queue.cc:1035] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:13.483729 7558 consensus_queue.cc:1035] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:13.486531 7562 mvcc.cc:205] Tried to move back new op lower bound from 7315542644664954880 to 7315542644621131776. Current Snapshot: MvccSnapshot[applied={T|T < 7315542644664954880}]
I20260806 12:59:13.486626 7561 mvcc.cc:205] Tried to move back new op lower bound from 7315542644664954880 to 7315542644621131776. Current Snapshot: MvccSnapshot[applied={T|T < 7315542644664954880 or (T in {7315542644664954880})}]
I20260806 12:59:13.486756 7563 mvcc.cc:205] Tried to move back new op lower bound from 7315542644664954880 to 7315542644621131776. Current Snapshot: MvccSnapshot[applied={T|T < 7315542644664954880}]
I20260806 12:59:13.488327 7077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:59:13.488411 7077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:59:13.489157 7077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Sending DeleteTablet for 3 replicas of tablet 0e16cfe5c67d4290ab0971348613b928
I20260806 12:59:13.489503 7473 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e16cfe5c67d4290ab0971348613b928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:13 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:53402
I20260806 12:59:13.489507 7338 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e16cfe5c67d4290ab0971348613b928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:13 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:48738
I20260806 12:59:13.489706 7194 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e16cfe5c67d4290ab0971348613b928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:13 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:57666
I20260806 12:59:13.489729 7571 tablet_replica.cc:330] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db: stopping tablet replica
I20260806 12:59:13.489812 7275 tablet_replica.cc:330] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: stopping tablet replica
I20260806 12:59:13.489821 7571 raft_consensus.cc:2229] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:59:13.489848 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 7116
I20260806 12:59:13.489885 7275 raft_consensus.cc:2229] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:59:13.489921 7571 raft_consensus.cc:2258] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:13.489936 7275 raft_consensus.cc:2258] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:13.491294 7571 ts_tablet_manager.cc:1905] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:59:13.493078 7572 tablet_replica.cc:330] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: stopping tablet replica
I20260806 12:59:13.493245 7572 raft_consensus.cc:2229] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:59:13.493417 7572 raft_consensus.cc:2258] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:13.493407 7571 ts_tablet_manager.cc:1918] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:13.493489 7571 log.cc:1196] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/0e16cfe5c67d4290ab0971348613b928
I20260806 12:59:13.493814 7571 ts_tablet_manager.cc:1939] T 0e16cfe5c67d4290ab0971348613b928 P b66d672365434791a0d2d43d60ce91db: Deleting consensus metadata
I20260806 12:59:13.495865 7572 ts_tablet_manager.cc:1905] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:59:13.496521 7061 catalog_manager.cc:4841] TS b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745): tablet 0e16cfe5c67d4290ab0971348613b928 (table test-table [id=32c4d5e694f046a79f98742382a49761]) successfully deleted
I20260806 12:59:13.497582 7572 ts_tablet_manager.cc:1918] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:13.497663 7572 log.cc:1196] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/0e16cfe5c67d4290ab0971348613b928
I20260806 12:59:13.497964 7572 ts_tablet_manager.cc:1939] T 0e16cfe5c67d4290ab0971348613b928 P d24eebe04e534ca6b37caf56fae9bbd7: Deleting consensus metadata
I20260806 12:59:13.498435 7061 catalog_manager.cc:4841] TS d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315): tablet 0e16cfe5c67d4290ab0971348613b928 (table test-table [id=32c4d5e694f046a79f98742382a49761]) successfully deleted
W20260806 12:59:13.500429 7061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:44351 (error 108)
I20260806 12:59:13.500748 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 7271
I20260806 12:59:13.507269 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 7409
I20260806 12:59:13.507406 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.507232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:44351 (local address 127.0.0.1:57682)
0806 12:59:13.507299 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.507367 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 12:59:13.509799 7061 catalog_manager.cc:4584] TS dc507127e3f64fd19e6c814055a0e35b (127.24.83.1:44351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e16cfe5c67d4290ab0971348613b928: Network error: Client connection negotiation failed: client connection to 127.24.83.1:44351: connect: Connection refused (error 111)
I20260806 12:59:13.513408 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 7043
I20260806 12:59:13.519121 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33939
--webserver_interface=127.24.83.62
--webserver_port=36633
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:33939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:13.600901 7574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:13.601084 7574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:13.601106 7574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:13.602200 7574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:59:13.602240 7574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:13.602253 7574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:13.602265 7574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:59:13.602277 7574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:59:13.603852 7574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:33939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=36633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:13.604100 7574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:13.604326 7574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:13.609046 7579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:13.609118 7580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:13.609129 7574 server_base.cc:1034] running on GCE node
W20260806 12:59:13.609074 7582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:13.609498 7574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:13.609789 7574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:13.610930 7574 hybrid_clock.cc:648] HybridClock initialized: now 1786021153610925 us; error 40 us; skew 500 ppm
Aug 06 12:59:13 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:13.613329 7574 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:59:13.613806 7574 webserver.cc:460] Webserver started at http://127.24.83.62:36633/ using document root <none> and password file <none>
I20260806 12:59:13.613996 7574 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:13.614037 7574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:13.615278 7574 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:59:13.615968 7589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.616169 7574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:59:13.616242 7574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "478b4505ddc5406595f298298d9ce226"
format_stamp: "Formatted at 2026-08-06 12:59:11 on dist-test-slave-zkpd"
server_key: "2a8393524e63944bde2e66468bad61b6"
server_key_iv: "5af0b83a053c3056405ba267afd323f0"
server_key_version: "encryptionkey@0"
I20260806 12:59:13.616545 7574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:13.624666 7574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:13.626735 7574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:13.626839 7574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:13.630904 7574 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:33939
I20260806 12:59:13.630932 7641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:33939 every 8 connection(s)
I20260806 12:59:13.631292 7574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:59:13.632887 7642 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:59:13.633464 7642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Bootstrap starting.
I20260806 12:59:13.634048 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 7574
I20260806 12:59:13.634342 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:44351
--local_ip_for_outbound_sockets=127.24.83.1
--tserver_master_addrs=127.24.83.62:33939
--webserver_port=44733
--webserver_interface=127.24.83.1
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:13.642935 7642 log.cc:824] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:13.645174 7642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:59:13.645483 7642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226: Bootstrap complete.
I20260806 12:59:13.647670 7642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:13.647922 7642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478b4505ddc5406595f298298d9ce226, State: Initialized, Role: FOLLOWER
I20260806 12:59:13.648109 7642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:13.648190 7642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:59:13.648242 7642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:59:13.648295 7642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 2 FOLLOWER]: Advancing to term 3
I20260806 12:59:13.649252 7642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:13.649381 7642 leader_election.cc:302] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 478b4505ddc5406595f298298d9ce226; no voters:
I20260806 12:59:13.649580 7642 leader_election.cc:288] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 12:59:13.649612 7646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 12:59:13.649717 7646 raft_consensus.cc:686] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [term 3 LEADER]: Becoming Leader. State: Replica: 478b4505ddc5406595f298298d9ce226, State: Running, Role: LEADER
I20260806 12:59:13.649813 7646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } }
I20260806 12:59:13.649849 7642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:59:13.650300 7648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 478b4505ddc5406595f298298d9ce226. Latest consensus state: current_term: 3 leader_uuid: "478b4505ddc5406595f298298d9ce226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } } }
I20260806 12:59:13.650382 7648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:13.650601 7650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:59:13.651064 7647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "478b4505ddc5406595f298298d9ce226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478b4505ddc5406595f298298d9ce226" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 33939 } } }
I20260806 12:59:13.651149 7647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 478b4505ddc5406595f298298d9ce226 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:13.651427 7650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:59:13.651539 7650 catalog_manager.cc:1259] Loaded cluster ID: 7559899162e9487f8050f0a3ac061f6d
I20260806 12:59:13.651736 7650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 12:59:13.652001 7650 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fba5d1619ad google::LogMessage::Fail() at ??:0
@ 0x7fba5d16488a google::LogMessage::SendToLog() at ??:0
@ 0x7fba5d1614de google::LogMessage::Flush() at ??:0
@ 0x7fba5d162a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fba650860f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fba6508f8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fba5f02f0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fba5f024954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fba5fe886db start_thread at ??:0
@ 0x7fba5bc5f61f clone at ??:0
W20260806 12:59:13.748307 7644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:13.748577 7644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:13.748615 7644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:13.750290 7644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:13.750350 7644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:13.750389 7644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:59:13.752645 7644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:44351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=44733
--tserver_master_addrs=127.24.83.62:33939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:13.752915 7644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:13.753183 7644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:13.758564 7681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:13.758545 7679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:13.758541 7678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:13.758826 7644 server_base.cc:1034] running on GCE node
I20260806 12:59:13.759069 7644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:13.759331 7644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:13.760494 7644 hybrid_clock.cc:648] HybridClock initialized: now 1786021153760460 us; error 49 us; skew 500 ppm
Aug 06 12:59:13 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:13.763193 7644 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:59:13.763765 7644 webserver.cc:460] Webserver started at http://127.24.83.1:44733/ using document root <none> and password file <none>
I20260806 12:59:13.764075 7644 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:13.764164 7644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:13.765937 7644 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:59:13.766698 7688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.766898 7644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:13.766981 7644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "dc507127e3f64fd19e6c814055a0e35b"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "f8a98ec0b1c820f9ebd839bd5dcc33b0"
server_key_iv: "34945ae537ac38b298de40f0d9d35964"
server_key_version: "encryptionkey@0"
I20260806 12:59:13.767328 7644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:13.778904 7644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:13.781370 7644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:13.781527 7644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:13.781800 7644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:13.782289 7696 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 12:59:13.783712 7644 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 12:59:13.783782 7644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 12:59:13.783821 7644 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 12:59:13.784746 7644 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 12:59:13.784812 7644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 12:59:13.784860 7696 tablet_bootstrap.cc:492] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: Bootstrap starting.
I20260806 12:59:13.792286 7644 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:44351
I20260806 12:59:13.792465 7803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:44351 every 8 connection(s)
I20260806 12:59:13.792721 7644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:59:13.795321 7696 log.cc:824] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:13.797468 7696 tablet_bootstrap.cc:492] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:59:13.797758 7696 tablet_bootstrap.cc:492] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: Bootstrap complete.
I20260806 12:59:13.798710 7696 ts_tablet_manager.cc:1397] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20260806 12:59:13.799705 7696 raft_consensus.cc:348] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.800005 7696 raft_consensus.cc:729] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc507127e3f64fd19e6c814055a0e35b, State: Initialized, Role: FOLLOWER
I20260806 12:59:13.800177 7696 consensus_queue.cc:260] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:13.800215 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 7644
I20260806 12:59:13.800457 7696 ts_tablet_manager.cc:1428] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260806 12:59:13.800500 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:44315
--local_ip_for_outbound_sockets=127.24.83.2
--tserver_master_addrs=127.24.83.62:33939
--webserver_port=38135
--webserver_interface=127.24.83.2
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:13.877508 7806 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:59:13.793479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:53167)
0806 12:59:13.793717 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.793773 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:13.794492 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:13.877294 (+ 82802us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":99,"threads_started":1}
W20260806 12:59:13.877820 7804 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:33939 (0 consecutive failures): Network error: Failed to ping master at 127.24.83.62:33939: Client connection negotiation failed: client connection to 127.24.83.62:33939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 12:59:13.878201 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.878069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:57739)
0806 12:59:13.878126 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.878173 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 12:59:13.878567 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.878465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:38261)
0806 12:59:13.878510 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.878548 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 12:59:13.878695 7804 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:59:13.883327 7809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:13.883499 7809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:13.883549 7809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:13.885282 7809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:13.885350 7809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:13.885401 7809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:59:13.886807 7809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:44315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=38135
--tserver_master_addrs=127.24.83.62:33939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:13.887061 7809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:13.887295 7809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:59:13.892277 7809 server_base.cc:1034] running on GCE node
W20260806 12:59:13.892266 7815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:13.892249 7817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:13.892261 7814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:13.892621 7809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:13.892876 7809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:13.894043 7809 hybrid_clock.cc:648] HybridClock initialized: now 1786021153894030 us; error 26 us; skew 500 ppm
Aug 06 12:59:13 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:13.896545 7809 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:59:13.896962 7809 webserver.cc:460] Webserver started at http://127.24.83.2:38135/ using document root <none> and password file <none>
I20260806 12:59:13.897199 7809 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:13.897269 7809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:13.898522 7809 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:13.899106 7824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.899281 7809 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:59:13.899385 7809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "d24eebe04e534ca6b37caf56fae9bbd7"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "cf14a3fd2f663b64adc2057bc3ddb81c"
server_key_iv: "f898fdfbf02c4d59c6c86c24e9ec2a66"
server_key_version: "encryptionkey@0"
I20260806 12:59:13.899708 7809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:13.926211 7809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:13.928109 7809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:13.928270 7809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:13.928506 7809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:13.928821 7809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:13.928875 7809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.928916 7809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:13.928944 7809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:13.934981 7809 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:44315
I20260806 12:59:13.935025 7938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:44315 every 8 connection(s)
I20260806 12:59:13.935356 7809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:59:13.935979 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 7809
I20260806 12:59:13.936249 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:35745
--local_ip_for_outbound_sockets=127.24.83.3
--tserver_master_addrs=127.24.83.62:33939
--webserver_port=42625
--webserver_interface=127.24.83.3
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:13.936663 7941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.936078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:53543)
0806 12:59:13.936252 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.936522 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
W20260806 12:59:13.937114 7939 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:33939 (0 consecutive failures): Network error: Failed to ping master at 127.24.83.62:33939: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
I20260806 12:59:13.937891 7941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.937778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:53245)
0806 12:59:13.937830 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.937859 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 12:59:13.938300 7941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:13.938198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:43005)
0806 12:59:13.938248 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:13.938275 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 12:59:13.938413 7939 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:59:14.017772 7942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:14.017961 7942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:14.017983 7942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:14.019033 7942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:14.019065 7942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:14.019089 7942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:59:14.020510 7942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:35745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=42625
--tserver_master_addrs=127.24.83.62:33939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:14.020704 7942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:14.020887 7942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:14.025889 7948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:14.025964 7950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:14.025905 7947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:14.026087 7942 server_base.cc:1034] running on GCE node
I20260806 12:59:14.026309 7942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:14.026535 7942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:14.027675 7942 hybrid_clock.cc:648] HybridClock initialized: now 1786021154027663 us; error 30 us; skew 500 ppm
Aug 06 12:59:14 dist-test-slave-zkpd krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021154, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:14.030097 7942 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:59:14.030556 7942 webserver.cc:460] Webserver started at http://127.24.83.3:42625/ using document root <none> and password file <none>
I20260806 12:59:14.030823 7942 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:14.030871 7942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:14.032637 7942 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 12:59:14.033234 7957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:14.033399 7942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:59:14.033499 7942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "b66d672365434791a0d2d43d60ce91db"
format_stamp: "Formatted at 2026-08-06 12:59:12 on dist-test-slave-zkpd"
server_key: "53bf9b5073132ffbe29a16c2c0c84c56"
server_key_iv: "b26639998f0b4728fd30ad32df02b9df"
server_key_version: "encryptionkey@0"
I20260806 12:59:14.033864 7942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:14.069900 7942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:14.071789 7942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:14.071997 7942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:14.072228 7942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:14.072537 7942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:14.072589 7942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:14.072623 7942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:14.072649 7942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:14.078308 7942 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:35745
I20260806 12:59:14.078413 8071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:35745 every 8 connection(s)
I20260806 12:59:14.078706 7942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:59:14.079785 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:14.079395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:42285)
0806 12:59:14.079549 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:14.079622 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
W20260806 12:59:14.080219 8072 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:33939 (0 consecutive failures): Network error: Failed to ping master at 127.24.83.62:33939: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
I20260806 12:59:14.080513 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:14.080412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:50869)
0806 12:59:14.080461 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:14.080489 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 12:59:14.080822 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:14.080745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:37519)
0806 12:59:14.080785 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:14.080806 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 12:59:14.080953 8072 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 12:59:14.082175 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 7942
I20260806 12:59:14.879249 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:14.879040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:38007)
0806 12:59:14.879154 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:14.879187 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
I20260806 12:59:14.938918 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:14.938698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:35819)
0806 12:59:14.938820 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:14.938856 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260806 12:59:14.951354 8078 raft_consensus.cc:482] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:59:14.951530 8078 raft_consensus.cc:504] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:14.951875 8078 leader_election.cc:288] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745), d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315)
Aug 06 12:59:14 dist-test-slave-zkpd krb5kdc[6488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.2@KRBTEST.COM
Aug 06 12:59:14 dist-test-slave-zkpd krb5kdc[6488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.3@KRBTEST.COM
I20260806 12:59:14.971050 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:14.952089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:35745 (local address 127.24.83.1:40697)
0806 12:59:14.952149 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:14.952161 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:14.952221 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:14.953323 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:14.953327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:14.953934 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:14.954246 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:14.954254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:14.955934 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:14.955935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:14.957362 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:14.957365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:14.958232 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:14.958233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:14.958384 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:14.965271 (+ 6887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:14.965283 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:14.967467 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:14.969519 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:14.969520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:14.969520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:14.969737 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:14.970061 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:14.970062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:14.970062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:14.970131 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:14.970545 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:14.970546 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:14.970646 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 12:59:14.970757 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 12:59:14.970956 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":98}
I20260806 12:59:14.971065 8081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:14.952353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:37923 (local address 127.24.83.2:44315)
0806 12:59:14.952486 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:14.952493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:14.952527 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:59:14.953024 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:14.953026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:14.953080 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:14.953235 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:14.954380 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:14.955758 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:14.957463 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:14.958120 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:14.958236 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:14.967209 (+ 8973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:14.967214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:14.967214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:14.967227 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:14.969090 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:14.969659 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:14.969662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:14.969663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:14.969781 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:14.970070 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:14.970071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:14.970071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:14.970414 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:14.970477 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:14.970814 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 12:59:14.970945 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260806 12:59:14.971066 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:14.952206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:40697 (local address 127.24.83.3:35745)
0806 12:59:14.952401 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:14.952415 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:14.952443 (+ 28us) server_negotiation.cc:407] Connection header received
0806 12:59:14.953001 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:14.953004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:14.953080 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:14.953236 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:14.954380 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:14.955783 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:14.957463 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:14.958120 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:14.958236 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:14.967584 (+ 9348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:14.967588 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:14.967589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:14.967601 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:14.969380 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:14.969851 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:14.969854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:14.969854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:14.969972 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:14.970198 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:14.970198 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:14.970198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:14.970438 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:14.970578 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:14.970819 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 12:59:14.970946 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260806 12:59:14.971189 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:14.952244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:44315 (local address 127.24.83.1:37923)
0806 12:59:14.952353 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:14.952364 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:14.952413 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:14.953323 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:14.953327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:14.953935 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:14.954245 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:14.954254 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:14.955900 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:14.955902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:14.957362 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:14.957365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:14.958234 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:14.958235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:14.958384 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:14.965271 (+ 6887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:14.965283 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:14.967091 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:14.969187 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:14.969195 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:14.969197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:14.969570 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:14.969853 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:14.969854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:14.969854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:14.970003 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:14.970492 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:14.970500 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:14.970646 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 12:59:14.970757 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 12:59:14.971041 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":198,"thread_start_us":54,"threads_started":1}
W20260806 12:59:14.972107 7690 leader_election.cc:341] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315): Not found: Tablet not found: 0e16cfe5c67d4290ab0971348613b928
W20260806 12:59:14.972788 7690 leader_election.cc:341] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745): Not found: Tablet not found: 0e16cfe5c67d4290ab0971348613b928
I20260806 12:59:14.972854 7690 leader_election.cc:302] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc507127e3f64fd19e6c814055a0e35b; no voters: b66d672365434791a0d2d43d60ce91db, d24eebe04e534ca6b37caf56fae9bbd7
I20260806 12:59:14.973006 8078 raft_consensus.cc:2726] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:59:15.081480 8082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:15.081260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:42045)
0806 12:59:15.081395 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:15.081439 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260806 12:59:15.879890 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:15.879668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:40309)
0806 12:59:15.879792 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:15.879841 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260806 12:59:15.939525 8084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:15.939335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:48261)
0806 12:59:15.939450 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:15.939486 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260806 12:59:16.082103 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:16.081898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:46261)
0806 12:59:16.082032 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:16.082066 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260806 12:59:16.880576 8086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:16.880365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:54029)
0806 12:59:16.880492 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:16.880527 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260806 12:59:16.940284 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:16.940072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:53159)
0806 12:59:16.940193 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:16.940234 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260806 12:59:17.082759 8088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:17.082552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:48181)
0806 12:59:17.082677 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:17.082718 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260806 12:59:17.093392 8089 raft_consensus.cc:482] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:59:17.093508 8089 raft_consensus.cc:504] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:17.093710 8089 leader_election.cc:288] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745), d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315)
W20260806 12:59:17.094173 7690 leader_election.cc:341] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745): Not found: Tablet not found: 0e16cfe5c67d4290ab0971348613b928
W20260806 12:59:17.094242 7690 leader_election.cc:341] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315): Not found: Tablet not found: 0e16cfe5c67d4290ab0971348613b928
I20260806 12:59:17.094259 7690 leader_election.cc:302] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc507127e3f64fd19e6c814055a0e35b; no voters: b66d672365434791a0d2d43d60ce91db, d24eebe04e534ca6b37caf56fae9bbd7
I20260806 12:59:17.094321 8089 raft_consensus.cc:2726] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:59:17.881314 8090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:17.881085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:58719)
0806 12:59:17.881227 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:17.881257 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":71,"threads_started":1}
I20260806 12:59:17.940965 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:17.940745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:57195)
0806 12:59:17.940882 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:17.940921 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260806 12:59:18.083482 8092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:18.083288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:45535)
0806 12:59:18.083409 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:18.083442 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260806 12:59:18.882068 8093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:18.881843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:55591)
0806 12:59:18.881993 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:18.882022 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260806 12:59:18.941586 8094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:18.941396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:39375)
0806 12:59:18.941509 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:18.941542 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260806 12:59:19.084247 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:19.084014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:49505)
0806 12:59:19.084173 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:19.084209 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
I20260806 12:59:19.630095 8096 raft_consensus.cc:482] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:59:19.630208 8096 raft_consensus.cc:504] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:19.630340 8096 leader_election.cc:288] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745), d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315)
W20260806 12:59:19.630749 7690 leader_election.cc:341] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745): Not found: Tablet not found: 0e16cfe5c67d4290ab0971348613b928
W20260806 12:59:19.630841 7690 leader_election.cc:341] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315): Not found: Tablet not found: 0e16cfe5c67d4290ab0971348613b928
I20260806 12:59:19.630893 7690 leader_election.cc:302] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc507127e3f64fd19e6c814055a0e35b; no voters: b66d672365434791a0d2d43d60ce91db, d24eebe04e534ca6b37caf56fae9bbd7
I20260806 12:59:19.630991 8096 raft_consensus.cc:2726] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:59:19.882696 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:19.882487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:40907)
0806 12:59:19.882621 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:19.882649 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260806 12:59:19.942173 8098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:19.941987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:56755)
0806 12:59:19.942098 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:19.942133 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260806 12:59:20.085000 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:20.084806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:47755)
0806 12:59:20.084933 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:20.084963 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260806 12:59:20.883359 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:20.883108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:60017)
0806 12:59:20.883252 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:20.883297 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260806 12:59:20.942767 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:20.942560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:43413)
0806 12:59:20.942682 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:20.942716 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260806 12:59:21.085690 8102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:21.085375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:52937)
0806 12:59:21.085601 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:21.085653 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":56,"threads_started":1}
I20260806 12:59:21.884142 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:21.883887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:40229)
0806 12:59:21.884057 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:21.884092 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
I20260806 12:59:21.943454 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:21.943209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:35891)
0806 12:59:21.943368 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:21.943408 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
I20260806 12:59:22.086300 8105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:22.086106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:44733)
0806 12:59:22.086226 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:22.086260 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260806 12:59:22.884810 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:22.884581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:49621)
0806 12:59:22.884726 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:22.884761 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260806 12:59:22.944092 8107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:22.943878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:43915)
0806 12:59:22.944020 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:22.944052 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260806 12:59:23.086930 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:23.086716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:33573)
0806 12:59:23.086850 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:23.086893 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260806 12:59:23.850572 8109 raft_consensus.cc:482] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:59:23.850672 8109 raft_consensus.cc:504] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66d672365434791a0d2d43d60ce91db" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35745 } } peers { permanent_uuid: "d24eebe04e534ca6b37caf56fae9bbd7" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 44315 } } peers { permanent_uuid: "dc507127e3f64fd19e6c814055a0e35b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44351 } }
I20260806 12:59:23.850860 8109 leader_election.cc:288] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745), d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315)
W20260806 12:59:23.851274 7690 leader_election.cc:341] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b66d672365434791a0d2d43d60ce91db (127.24.83.3:35745): Not found: Tablet not found: 0e16cfe5c67d4290ab0971348613b928
W20260806 12:59:23.851372 7690 leader_election.cc:341] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d24eebe04e534ca6b37caf56fae9bbd7 (127.24.83.2:44315): Not found: Tablet not found: 0e16cfe5c67d4290ab0971348613b928
I20260806 12:59:23.851401 7690 leader_election.cc:302] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc507127e3f64fd19e6c814055a0e35b; no voters: b66d672365434791a0d2d43d60ce91db, d24eebe04e534ca6b37caf56fae9bbd7
I20260806 12:59:23.851475 8109 raft_consensus.cc:2726] T 0e16cfe5c67d4290ab0971348613b928 P dc507127e3f64fd19e6c814055a0e35b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:59:23.885474 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:23.885261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:57961)
0806 12:59:23.885396 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:23.885428 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260806 12:59:23.944765 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:23.944532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:57155)
0806 12:59:23.944687 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:23.944716 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260806 12:59:24.087648 8112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:24.087452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:43549)
0806 12:59:24.087564 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:24.087596 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260806 12:59:24.886268 8113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:24.886053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:36525)
0806 12:59:24.886185 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:24.886221 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260806 12:59:24.945376 8114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:24.945175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:60217)
0806 12:59:24.945297 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:24.945328 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260806 12:59:25.088284 8115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:25.088092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:59621)
0806 12:59:25.088207 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:25.088240 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260806 12:59:25.886898 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:25.886692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:55093)
0806 12:59:25.886816 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:25.886848 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260806 12:59:25.946055 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:25.945753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:47079)
0806 12:59:25.945974 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:25.946010 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":75,"threads_started":1}
I20260806 12:59:26.088864 8118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:26.088649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:59461)
0806 12:59:26.088790 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:26.088825 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260806 12:59:26.887569 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:26.887351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:35223)
0806 12:59:26.887484 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:26.887521 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260806 12:59:26.946684 8120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:26.946475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:43765)
0806 12:59:26.946608 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:26.946641 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
I20260806 12:59:27.089529 8121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:27.089295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:40193)
0806 12:59:27.089440 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:27.089470 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260806 12:59:27.888391 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:27.888151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:34367)
0806 12:59:27.888306 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:27.888342 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260806 12:59:27.947347 8123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:27.947137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:34383)
0806 12:59:27.947268 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:27.947298 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260806 12:59:28.090441 8124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:28.090143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:39651)
0806 12:59:28.090359 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:28.090396 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":79,"threads_started":1}
I20260806 12:59:28.889088 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:28.888890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.1:56659)
0806 12:59:28.889009 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:28.889041 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260806 12:59:28.948092 8126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:28.947832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.2:40873)
0806 12:59:28.948019 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:28.948051 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":136,"threads_started":1}
I20260806 12:59:29.091252 8127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:29.091028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:33939 (local address 127.24.83.3:58399)
0806 12:59:29.091179 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:29.091213 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":61,"threads_started":1}
I20260806 12:59:29.092134 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 7644
I20260806 12:59:29.099407 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 7809
I20260806 12:59:29.105365 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 7942
2026-08-06T12:59:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18515 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:59:29 dist-test-slave-zkpd krb5kdc[8130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:59:29 dist-test-slave-zkpd krb5kdc[8130](info): set up 2 sockets
Aug 06 12:59:29 dist-test-slave-zkpd krb5kdc[8130](info): commencing operation
krb5kdc: starting...
W20260806 12:59:30.156306 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 12:59:30 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:59:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:59:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 12:59:30.274745 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:34107
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:34107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:30.356374 8143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:30.356567 8143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:30.356590 8143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:30.357640 8143 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 12:59:30.357671 8143 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 12:59:30.357683 8143 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 12:59:30.357695 8143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:59:30.357707 8143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:30.357720 8143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:30.357744 8143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:59:30.357756 8143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:59:30.359128 8143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:34107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:34107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:30.359311 8143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:30.359479 8143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:30.364771 8148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:30.364777 8149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:30.364787 8151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:30.364928 8143 server_base.cc:1034] running on GCE node
I20260806 12:59:30.365207 8143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:30.365492 8143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:30.366653 8143 hybrid_clock.cc:648] HybridClock initialized: now 1786021170366630 us; error 35 us; skew 500 ppm
Aug 06 12:59:30 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021170, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:30.369190 8143 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:59:30.369632 8143 webserver.cc:460] Webserver started at http://127.24.83.62:41417/ using document root <none> and password file <none>
I20260806 12:59:30.369855 8143 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:30.369922 8143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:30.370067 8143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:30.371078 8143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "049b7e608ae54aefafae8104c741fec1"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "8ede9399ba21bd54da1527d08362fa7d"
server_key_iv: "533e38db3210dc18c2cfa0eecd35f211"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.371441 8143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "049b7e608ae54aefafae8104c741fec1"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "8ede9399ba21bd54da1527d08362fa7d"
server_key_iv: "533e38db3210dc18c2cfa0eecd35f211"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.372865 8143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 12:59:30.373584 8159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:30.373766 8143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:59:30.373890 8143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "049b7e608ae54aefafae8104c741fec1"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "8ede9399ba21bd54da1527d08362fa7d"
server_key_iv: "533e38db3210dc18c2cfa0eecd35f211"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.373962 8143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:30.385125 8143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:30.387401 8143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:30.387560 8143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:30.391611 8143 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:34107
I20260806 12:59:30.391649 8212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:34107 every 8 connection(s)
I20260806 12:59:30.392078 8143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:59:30.392964 8213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:30.395324 8213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Bootstrap starting.
I20260806 12:59:30.395947 8213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:30.396245 8213 log.cc:824] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:30.396919 8213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: No bootstrap required, opened a new log
I20260806 12:59:30.398486 8213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 12:59:30.398620 8213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:30.398643 8213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049b7e608ae54aefafae8104c741fec1, State: Initialized, Role: FOLLOWER
I20260806 12:59:30.398811 8213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 12:59:30.398895 8213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:59:30.398938 8213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:59:30.398988 8213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:30.399223 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 8143
I20260806 12:59:30.399331 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 12:59:30.399541 24908 external_mini_cluster.cc:1428] Setting key a4f4b9b3900b977ef03f0dfaa948d057
I20260806 12:59:30.399717 8213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 12:59:30.399845 8213 leader_election.cc:302] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 049b7e608ae54aefafae8104c741fec1; no voters:
I20260806 12:59:30.400038 8213 leader_election.cc:288] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:59:30.400063 8216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:59:30.400269 8216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 1 LEADER]: Becoming Leader. State: Replica: 049b7e608ae54aefafae8104c741fec1, State: Running, Role: LEADER
I20260806 12:59:30.400360 8213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:59:30.400359 8216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 12:59:30.400993 8219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 049b7e608ae54aefafae8104c741fec1. Latest consensus state: current_term: 1 leader_uuid: "049b7e608ae54aefafae8104c741fec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } } }
I20260806 12:59:30.401075 8219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:30.401315 8218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "049b7e608ae54aefafae8104c741fec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } } }
I20260806 12:59:30.401412 8218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:30.401568 8227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:59:30.402081 8227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:59:30.403800 8227 catalog_manager.cc:1347] Generated new cluster ID: 0a48e20e0d6d4e74afedac5fb1eec1ac
I20260806 12:59:30.403803 8234 mvcc.cc:205] Tried to move back new op lower bound from 7315542713970565120 to 7315542713961156608. Current Snapshot: MvccSnapshot[applied={T|T < 7315542713970565120 or (T in {7315542713970565120})}]
I20260806 12:59:30.403855 8227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 12:59:30 dist-test-slave-zkpd krb5kdc[8130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:30.411453 8220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.400078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.24.83.62:34107)
0806 12:59:30.400773 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:30.400781 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:30.400812 (+ 31us) server_negotiation.cc:407] Connection header received
0806 12:59:30.400923 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:30.400929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:30.400982 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:30.401173 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:30.402009 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.403119 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.404207 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.404792 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.404876 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.407810 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:30.407814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:30.407814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:30.407826 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:30.409815 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:30.410219 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:30.410222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:30.410223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:30.410327 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:30.410563 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:30.410564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:30.410564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:30.410839 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:30.410954 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:30.411121 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 12:59:30.411247 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":1773,"threads_started":1}
I20260806 12:59:30.414206 8227 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:59:30.414872 8227 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:59:30.433167 8227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Generated new TSK 0
I20260806 12:59:30.433360 8227 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 12:59:30.462878 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:34107
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:30.544193 8239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:30.544456 8239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:30.544493 8239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:30.545644 8239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:30.545710 8239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:30.545754 8239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:59:30.547200 8239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:34107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:30.547465 8239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:30.547683 8239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:30.552680 8244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:30.552852 8239 server_base.cc:1034] running on GCE node
W20260806 12:59:30.552696 8247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:30.552696 8245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:30.553129 8239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:30.553375 8239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:30.554546 8239 hybrid_clock.cc:648] HybridClock initialized: now 1786021170554533 us; error 42 us; skew 500 ppm
Aug 06 12:59:30 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021170, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:30.557036 8239 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:59:30.557500 8239 webserver.cc:460] Webserver started at http://127.24.83.1:41183/ using document root <none> and password file <none>
I20260806 12:59:30.557716 8239 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:30.557782 8239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:30.557933 8239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:30.558923 8239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "7fff71f6435f4c2c91ea52bc0931259f"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "371a1815ee82edfb78a5353e942181f0"
server_key_iv: "d047f18e8bd1a5aa8163b7ee98a225ff"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.559280 8239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "7fff71f6435f4c2c91ea52bc0931259f"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "371a1815ee82edfb78a5353e942181f0"
server_key_iv: "d047f18e8bd1a5aa8163b7ee98a225ff"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.560667 8239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:59:30.561422 8255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:30.561616 8239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:30.561721 8239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "7fff71f6435f4c2c91ea52bc0931259f"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "371a1815ee82edfb78a5353e942181f0"
server_key_iv: "d047f18e8bd1a5aa8163b7ee98a225ff"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.561794 8239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:30.603741 8239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:30.605610 8239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:30.605779 8239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:30.606019 8239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:30.606345 8239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:30.606402 8239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:30.606439 8239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:30.606467 8239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:30.612347 8239 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:46053
I20260806 12:59:30.612403 8371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:46053 every 8 connection(s)
I20260806 12:59:30.612704 8239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:59:30.618237 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 8239
I20260806 12:59:30.618336 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 12:59:30.618567 24908 external_mini_cluster.cc:1428] Setting key 1d30323fc4a8c7d1528f1f14be0babda
Aug 06 12:59:30 dist-test-slave-zkpd krb5kdc[8130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021170, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:30.622498 8220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.613410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:58975 (local address 127.24.83.62:34107)
0806 12:59:30.613497 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:30.613500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:30.613689 (+ 189us) server_negotiation.cc:407] Connection header received
0806 12:59:30.614160 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:30.614162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:30.614172 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:30.614219 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:30.615155 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.615655 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.616681 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.616883 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.616938 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.619380 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:30.619381 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:30.619382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:30.619389 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:30.621074 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:30.621596 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:30.621597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:30.621597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:30.621647 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:30.621926 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:30.621927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:30.621927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:30.622024 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:30.622074 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:30.622373 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 12:59:30.622435 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:59:30.622699 8374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.613358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:58975)
0806 12:59:30.613544 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:30.613579 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:30.614073 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:30.614282 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:30.614285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:30.614853 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:30.615059 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.615064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.615735 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.615735 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.616603 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.616605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.616959 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.616960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.617063 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.617600 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:30.617609 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:30.619283 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:30.621165 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:30.621173 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:30.621176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:30.621513 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:30.621709 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:30.621710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:30.621710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:30.621861 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:30.622088 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:30.622094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:30.622245 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 12:59:30.622345 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 12:59:30.622462 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260806 12:59:30.623238 8372 heartbeater.cc:347] Connected to a master server at 127.24.83.62:34107
I20260806 12:59:30.623343 8372 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:30.623518 8372 heartbeater.cc:511] Master 127.24.83.62:34107 requested a full tablet report, sending...
I20260806 12:59:30.624097 8176 ts_manager.cc:194] Registered new tserver with Master: 7fff71f6435f4c2c91ea52bc0931259f (127.24.83.1:46053)
I20260806 12:59:30.625092 8176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:58975
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 12:59:30.646190 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:34107
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:30.727573 8377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:30.727802 8377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:30.727843 8377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:30.729025 8377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:30.729095 8377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:30.729140 8377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:59:30.730648 8377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:34107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:30.730940 8377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:30.731169 8377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:30.736145 8382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:30.736150 8385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:30.736150 8383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:30.736442 8377 server_base.cc:1034] running on GCE node
I20260806 12:59:30.736662 8377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:30.736917 8377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:30.738085 8377 hybrid_clock.cc:648] HybridClock initialized: now 1786021170738072 us; error 29 us; skew 500 ppm
Aug 06 12:59:30 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021170, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:30.740586 8377 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:59:30.740999 8377 webserver.cc:460] Webserver started at http://127.24.83.2:41027/ using document root <none> and password file <none>
I20260806 12:59:30.741217 8377 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:30.741262 8377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:30.741403 8377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:30.742429 8377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "9638e09612a944cab7e51f95a2016037"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "dd04611faec26138ef66d56a111c69e0"
server_key_iv: "ac03737a1c36690f02429f7b8caf5f4f"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.742782 8377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "9638e09612a944cab7e51f95a2016037"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "dd04611faec26138ef66d56a111c69e0"
server_key_iv: "ac03737a1c36690f02429f7b8caf5f4f"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.744138 8377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 12:59:30.744875 8392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:30.745067 8377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:59:30.745138 8377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "9638e09612a944cab7e51f95a2016037"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "dd04611faec26138ef66d56a111c69e0"
server_key_iv: "ac03737a1c36690f02429f7b8caf5f4f"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.745227 8377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:30.766279 8377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:30.768242 8377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:30.768381 8377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:30.768615 8377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:30.768919 8377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:30.768975 8377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:30.769016 8377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:30.769043 8377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:30.775197 8377 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:45305
I20260806 12:59:30.775247 8507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:45305 every 8 connection(s)
I20260806 12:59:30.775553 8377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 12:59:30.780939 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 8377
I20260806 12:59:30.781062 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 12:59:30.781263 24908 external_mini_cluster.cc:1428] Setting key f72e4b3584e84b12c54cff403b3643ca
Aug 06 12:59:30 dist-test-slave-zkpd krb5kdc[8130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021170, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:30.785225 8220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.776259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:33861 (local address 127.24.83.62:34107)
0806 12:59:30.776306 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:30.776308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:30.776489 (+ 181us) server_negotiation.cc:407] Connection header received
0806 12:59:30.776977 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:30.776978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:30.777022 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:30.777083 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:30.777967 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.778467 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.779456 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.779640 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.779689 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.782162 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:30.782164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:30.782164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:30.782172 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:30.783768 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:30.784277 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:30.784277 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:30.784278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:30.784328 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:30.784616 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:30.784617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:30.784617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:30.784735 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:30.784791 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:30.785094 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 12:59:30.785162 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 12:59:30.785418 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.776236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:33861)
0806 12:59:30.776374 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:30.776408 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:30.776891 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:30.777151 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:30.777154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:30.777702 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:30.777892 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.777897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.778551 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.778551 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.779363 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.779364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.779713 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.779713 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.779832 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.780403 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:30.780411 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:30.782069 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:30.783858 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:30.783866 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:30.783868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:30.784198 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:30.784408 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:30.784409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:30.784410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:30.784551 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:30.784808 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:30.784814 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:30.784980 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 12:59:30.785067 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 12:59:30.785184 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260806 12:59:30.785876 8508 heartbeater.cc:347] Connected to a master server at 127.24.83.62:34107
I20260806 12:59:30.785984 8508 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:30.786165 8508 heartbeater.cc:511] Master 127.24.83.62:34107 requested a full tablet report, sending...
I20260806 12:59:30.786512 8176 ts_manager.cc:194] Registered new tserver with Master: 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305)
I20260806 12:59:30.786978 8176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:33861
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 12:59:30.808054 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:34107
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:30.888942 8513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:30.889135 8513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:30.889158 8513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:30.890306 8513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:30.890345 8513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:30.890372 8513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:59:30.891789 8513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:34107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:30.892055 8513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:30.892289 8513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:30.897111 8518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:30.897116 8519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:30.897117 8521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:30.897133 8513 server_base.cc:1034] running on GCE node
I20260806 12:59:30.897569 8513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:30.897809 8513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:30.898947 8513 hybrid_clock.cc:648] HybridClock initialized: now 1786021170898935 us; error 30 us; skew 500 ppm
Aug 06 12:59:30 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021170, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:30.901829 8513 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:59:30.902276 8513 webserver.cc:460] Webserver started at http://127.24.83.3:36225/ using document root <none> and password file <none>
I20260806 12:59:30.902508 8513 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:30.902550 8513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:30.902698 8513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:59:30.903699 8513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "6e68537becb04c1c99d4981ffa922bd1"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "ad87e76e9eef9bc4e8458cf33d498ac9"
server_key_iv: "c26e62653aafbcb1084b55e650cbccdc"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.904086 8513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "6e68537becb04c1c99d4981ffa922bd1"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "ad87e76e9eef9bc4e8458cf33d498ac9"
server_key_iv: "c26e62653aafbcb1084b55e650cbccdc"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.905367 8513 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:59:30.906047 8528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:30.906222 8513 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 12:59:30.906293 8513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "6e68537becb04c1c99d4981ffa922bd1"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "ad87e76e9eef9bc4e8458cf33d498ac9"
server_key_iv: "c26e62653aafbcb1084b55e650cbccdc"
server_key_version: "encryptionkey@0"
I20260806 12:59:30.906388 8513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:30.922748 8513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:30.924957 8513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:30.925113 8513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:30.925379 8513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:30.925709 8513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:30.925742 8513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:30.925763 8513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:30.925808 8513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:30.931672 8513 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:43223
I20260806 12:59:30.931716 8644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:43223 every 8 connection(s)
I20260806 12:59:30.932070 8513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:59:30.932327 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 8513
I20260806 12:59:30.932428 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 12:59:30.932650 24908 external_mini_cluster.cc:1428] Setting key 87adcd44b4c5b1eec26fa6d91763a0e3
Aug 06 12:59:30 dist-test-slave-zkpd krb5kdc[8130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021170, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:30.941022 8220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.932796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:55433 (local address 127.24.83.62:34107)
0806 12:59:30.932904 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:30.932906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:30.933109 (+ 203us) server_negotiation.cc:407] Connection header received
0806 12:59:30.933781 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:30.933782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:30.933793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:30.933841 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:30.934736 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.935225 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.936254 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.936438 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.936492 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.938420 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:30.938421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:30.938422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:30.938429 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:30.939818 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:30.940216 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:30.940216 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:30.940217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:30.940267 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:30.940472 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:30.940473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:30.940473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:30.940570 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:30.940619 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:30.940873 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 12:59:30.940945 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 12:59:30.941133 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.932791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:55433)
0806 12:59:30.932983 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:30.933030 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:30.933658 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:30.933924 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:30.933927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:30.934475 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:30.934660 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.934665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.935303 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.935304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.936185 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.936187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.936510 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.936510 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.936606 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.937039 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:30.937047 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:30.938332 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:30.939944 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:30.939950 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:30.939952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:30.940153 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:30.940327 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:30.940328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:30.940328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:30.940418 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:30.940623 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:30.940627 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:30.940730 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 12:59:30.940808 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:59:30.940929 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260806 12:59:30.941625 8645 heartbeater.cc:347] Connected to a master server at 127.24.83.62:34107
I20260806 12:59:30.941722 8645 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:30.941890 8645 heartbeater.cc:511] Master 127.24.83.62:34107 requested a full tablet report, sending...
I20260806 12:59:30.942250 8176 ts_manager.cc:194] Registered new tserver with Master: 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223)
I20260806 12:59:30.942584 8176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:55433
I20260806 12:59:30.944415 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:59:30.951366 8220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.945655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34126 (local address 127.24.83.62:34107)
0806 12:59:30.945712 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:30.945714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:30.945823 (+ 109us) server_negotiation.cc:407] Connection header received
0806 12:59:30.945896 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:30.945897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:30.945908 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:30.945961 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:30.946508 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.946964 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.947687 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.947950 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.948039 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.949117 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:30.949118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:30.949118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:30.949125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:30.950250 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:30.950587 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:30.950588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:30.950589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:30.950641 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:30.950841 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:30.950842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:30.950842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:30.950984 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:30.951087 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:30.951194 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 12:59:30.951268 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 12:59:30.952790 8176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:59:30.954285 8176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:59:30.960521 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.956479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:46053 (local address 127.0.0.1:40498)
0806 12:59:30.956848 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:30.956857 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:30.956933 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:30.957120 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:30.957123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:30.957126 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:30.957187 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.957189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.958109 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.958110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.958957 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.958959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.960230 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.960231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.960319 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.960331 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 12:59:30.960394 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 12:59:30.960446 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":3,"thread_start_us":86,"threads_started":1}
I20260806 12:59:30.960812 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.956596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40498 (local address 127.24.83.1:46053)
0806 12:59:30.956805 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:30.956810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:30.956938 (+ 128us) server_negotiation.cc:407] Connection header received
0806 12:59:30.956997 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:30.957000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:30.957015 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:30.957112 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:30.957285 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.957985 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.959290 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.960076 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.960145 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.960197 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:30.960677 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 12:59:30.960732 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":86,"threads_started":1}
I20260806 12:59:30.960815 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.956479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:45305 (local address 127.0.0.1:45988)
0806 12:59:30.956626 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:30.956662 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:30.956739 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:30.957230 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:30.957231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:30.957233 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:30.957276 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.957277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.958603 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.958604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.959677 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.959680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.960632 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.960634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.960693 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.960697 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:30.960726 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 12:59:30.960758 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":46,"threads_started":1}
I20260806 12:59:30.960927 8657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.956656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:43223 (local address 127.0.0.1:34296)
0806 12:59:30.956883 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:30.956894 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:30.956949 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:30.957544 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:30.957544 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:30.957546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:30.957594 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.957595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.958888 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.958889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.959620 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:30.959622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.960677 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.960678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:30.960745 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.960750 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:59:30.960814 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 12:59:30.960855 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":78,"threads_started":1}
I20260806 12:59:30.961130 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.956895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.24.83.3:43223)
0806 12:59:30.957070 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:30.957074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:30.957084 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:59:30.957426 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:30.957429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:30.957443 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:30.957676 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:30.957733 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.958787 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.959710 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.960503 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.960582 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.960638 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:30.960993 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 12:59:30.961038 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
I20260806 12:59:30.961790 8304 tablet_service.cc:1467] Processing CreateTablet for tablet fcac4403e6674291b124d2c69af56e0a (DEFAULT_TABLE table=test-table [id=3911ae981cb543409960e51bf2fc5259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:30.961841 8577 tablet_service.cc:1467] Processing CreateTablet for tablet fcac4403e6674291b124d2c69af56e0a (DEFAULT_TABLE table=test-table [id=3911ae981cb543409960e51bf2fc5259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:30.962138 8304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcac4403e6674291b124d2c69af56e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:30.962164 8577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcac4403e6674291b124d2c69af56e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:30.964970 8662 tablet_bootstrap.cc:492] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1: Bootstrap starting.
I20260806 12:59:30.965174 8663 tablet_bootstrap.cc:492] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Bootstrap starting.
I20260806 12:59:30.965235 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:30.956597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45988 (local address 127.24.83.2:45305)
0806 12:59:30.957079 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:30.957083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:30.957092 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:59:30.957132 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:30.957134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:30.957150 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:30.957224 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:30.957340 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.958487 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.959779 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:30.960531 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:30.960884 (+ 353us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:30.961215 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:30.961246 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 12:59:30.965133 (+ 3887us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":89,"threads_started":1}
I20260806 12:59:30.965730 8662 tablet_bootstrap.cc:654] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:30.965888 8663 tablet_bootstrap.cc:654] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:30.965960 8441 tablet_service.cc:1467] Processing CreateTablet for tablet fcac4403e6674291b124d2c69af56e0a (DEFAULT_TABLE table=test-table [id=3911ae981cb543409960e51bf2fc5259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:30.966313 8662 log.cc:824] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:30.966333 8441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcac4403e6674291b124d2c69af56e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:30.966462 8663 log.cc:824] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:30.966974 8662 tablet_bootstrap.cc:492] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1: No bootstrap required, opened a new log
I20260806 12:59:30.967025 8662 ts_tablet_manager.cc:1397] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:59:30.968586 8662 raft_consensus.cc:348] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:30.968717 8662 raft_consensus.cc:374] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:30.968739 8662 raft_consensus.cc:729] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e68537becb04c1c99d4981ffa922bd1, State: Initialized, Role: FOLLOWER
I20260806 12:59:30.968856 8662 consensus_queue.cc:260] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:30.969129 8666 tablet_bootstrap.cc:492] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: Bootstrap starting.
I20260806 12:59:30.969151 8662 ts_tablet_manager.cc:1428] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:59:30.969270 8645 heartbeater.cc:503] Master 127.24.83.62:34107 was elected leader, sending a full tablet report...
I20260806 12:59:30.969767 8666 tablet_bootstrap.cc:654] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:30.970081 8663 tablet_bootstrap.cc:492] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: No bootstrap required, opened a new log
I20260806 12:59:30.970175 8663 ts_tablet_manager.cc:1397] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 12:59:30.970331 8666 log.cc:824] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:30.971007 8666 tablet_bootstrap.cc:492] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: No bootstrap required, opened a new log
I20260806 12:59:30.971050 8666 ts_tablet_manager.cc:1397] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:59:30.971704 8663 raft_consensus.cc:348] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:30.971829 8663 raft_consensus.cc:374] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:30.971850 8663 raft_consensus.cc:729] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fff71f6435f4c2c91ea52bc0931259f, State: Initialized, Role: FOLLOWER
I20260806 12:59:30.971988 8663 consensus_queue.cc:260] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:30.972301 8663 ts_tablet_manager.cc:1428] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:59:30.972381 8372 heartbeater.cc:503] Master 127.24.83.62:34107 was elected leader, sending a full tablet report...
I20260806 12:59:30.972808 8666 raft_consensus.cc:348] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:30.973012 8666 raft_consensus.cc:374] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:30.973068 8666 raft_consensus.cc:729] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9638e09612a944cab7e51f95a2016037, State: Initialized, Role: FOLLOWER
I20260806 12:59:30.973165 8666 consensus_queue.cc:260] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:30.973479 8666 ts_tablet_manager.cc:1428] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 12:59:30.974130 8508 heartbeater.cc:503] Master 127.24.83.62:34107 was elected leader, sending a full tablet report...
I20260806 12:59:31.018553 8670 raft_consensus.cc:482] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:59:31.018682 8670 raft_consensus.cc:504] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:31.019110 8670 leader_election.cc:288] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223), 7fff71f6435f4c2c91ea52bc0931259f (127.24.83.1:46053)
I20260806 12:59:31.022097 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:31.019385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:50309 (local address 127.24.83.1:46053)
0806 12:59:31.019455 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:31.019457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:31.019466 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:59:31.019489 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:31.019490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:31.019499 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:31.019554 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:31.019751 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:31.020371 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:31.021337 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:31.021729 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:31.021822 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:31.021833 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:31.021995 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 12:59:31.022038 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:59:31.022094 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:31.019286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:46053 (local address 127.24.83.2:50309)
0806 12:59:31.019330 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:31.019341 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:31.019396 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:31.019580 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:31.019581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:31.019583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:31.019666 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:31.019668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:31.020470 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:31.020471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:31.021239 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:31.021241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:31.021820 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:31.021821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:31.021908 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:31.021911 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:59:31.021975 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 12:59:31.022018 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":21}
I20260806 12:59:31.022172 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:31.019388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:58045 (local address 127.24.83.3:43223)
0806 12:59:31.019455 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:31.019458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:31.019759 (+ 301us) server_negotiation.cc:407] Connection header received
0806 12:59:31.019828 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:31.019829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:31.019839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:31.019876 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:31.020042 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:31.020493 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:31.021433 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:31.021811 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:31.021901 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:31.021912 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:31.022062 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 12:59:31.022102 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:59:31.022228 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:31.019286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:43223 (local address 127.24.83.2:58045)
0806 12:59:31.019681 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:31.019690 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:31.019778 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:31.019882 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:31.019882 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:31.019884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:31.019975 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:31.019976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:31.020583 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:31.020584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:31.021325 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:31.021327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:31.021898 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:31.021898 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:31.021969 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:31.021972 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:59:31.022042 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 12:59:31.022080 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":182,"thread_start_us":139,"threads_started":1}
I20260806 12:59:31.022500 8324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcac4403e6674291b124d2c69af56e0a" candidate_uuid: "9638e09612a944cab7e51f95a2016037" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fff71f6435f4c2c91ea52bc0931259f" is_pre_election: true
I20260806 12:59:31.022491 8597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcac4403e6674291b124d2c69af56e0a" candidate_uuid: "9638e09612a944cab7e51f95a2016037" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68537becb04c1c99d4981ffa922bd1" is_pre_election: true
I20260806 12:59:31.022632 8597 raft_consensus.cc:2454] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9638e09612a944cab7e51f95a2016037 in term 0.
I20260806 12:59:31.022655 8324 raft_consensus.cc:2454] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9638e09612a944cab7e51f95a2016037 in term 0.
I20260806 12:59:31.022850 8393 leader_election.cc:302] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e68537becb04c1c99d4981ffa922bd1, 9638e09612a944cab7e51f95a2016037; no voters:
I20260806 12:59:31.022984 8670 raft_consensus.cc:2781] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:59:31.023041 8670 raft_consensus.cc:482] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:59:31.023066 8670 raft_consensus.cc:3037] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:31.023780 8670 raft_consensus.cc:504] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:31.023953 8670 leader_election.cc:288] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [CANDIDATE]: Term 1 election: Requested vote from peers 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223), 7fff71f6435f4c2c91ea52bc0931259f (127.24.83.1:46053)
I20260806 12:59:31.024112 8597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcac4403e6674291b124d2c69af56e0a" candidate_uuid: "9638e09612a944cab7e51f95a2016037" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68537becb04c1c99d4981ffa922bd1"
I20260806 12:59:31.024191 8597 raft_consensus.cc:3037] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:31.024158 8324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcac4403e6674291b124d2c69af56e0a" candidate_uuid: "9638e09612a944cab7e51f95a2016037" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fff71f6435f4c2c91ea52bc0931259f"
I20260806 12:59:31.024276 8324 raft_consensus.cc:3037] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:31.024904 8597 raft_consensus.cc:2454] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9638e09612a944cab7e51f95a2016037 in term 1.
I20260806 12:59:31.025025 8324 raft_consensus.cc:2454] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9638e09612a944cab7e51f95a2016037 in term 1.
I20260806 12:59:31.025069 8393 leader_election.cc:302] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e68537becb04c1c99d4981ffa922bd1, 9638e09612a944cab7e51f95a2016037; no voters:
I20260806 12:59:31.025141 8670 raft_consensus.cc:2781] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:59:31.025305 8670 raft_consensus.cc:686] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 1 LEADER]: Becoming Leader. State: Replica: 9638e09612a944cab7e51f95a2016037, State: Running, Role: LEADER
I20260806 12:59:31.025422 8670 consensus_queue.cc:237] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
W20260806 12:59:31.026022 8509 tablet.cc:2369] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:59:31.026222 8176 catalog_manager.cc:5495] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9638e09612a944cab7e51f95a2016037 (127.24.83.2). New cstate: current_term: 1 leader_uuid: "9638e09612a944cab7e51f95a2016037" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } health_report { overall_health: UNKNOWN } } }
I20260806 12:59:31.032994 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:31.030674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45994 (local address 127.24.83.2:45305)
0806 12:59:31.030729 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:31.030731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:31.030740 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:59:31.030793 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:31.030794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:31.030813 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:31.030903 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:59:31.031036 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:31.031482 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:31.032287 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:31.032471 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:31.032542 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:31.032703 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:59:31.032793 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:59:31.032881 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:31.032905 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 12:59:31.032921 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 12:59:31.034664 8597 raft_consensus.cc:1261] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1 [term 1 FOLLOWER]: Refusing update from remote peer 9638e09612a944cab7e51f95a2016037: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:31.034732 8324 raft_consensus.cc:1261] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Refusing update from remote peer 9638e09612a944cab7e51f95a2016037: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:31.035060 8672 consensus_queue.cc:1035] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:31.035212 8670 consensus_queue.cc:1035] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:31.037638 8675 mvcc.cc:205] Tried to move back new op lower bound from 7315542716555190272 to 7315542716520910848. Current Snapshot: MvccSnapshot[applied={T|T < 7315542716555190272}]
I20260806 12:59:31.038086 8676 mvcc.cc:205] Tried to move back new op lower bound from 7315542716555190272 to 7315542716520910848. Current Snapshot: MvccSnapshot[applied={T|T < 7315542716555190272}]
I20260806 12:59:31.040107 8175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:59:31.040195 8175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:59:31.040928 8175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Sending DeleteTablet for 3 replicas of tablet fcac4403e6674291b124d2c69af56e0a
I20260806 12:59:31.041263 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 8239
I20260806 12:59:31.041252 8304 tablet_service.cc:1514] Processing DeleteTablet for tablet fcac4403e6674291b124d2c69af56e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:31 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:40498
I20260806 12:59:31.041260 8441 tablet_service.cc:1514] Processing DeleteTablet for tablet fcac4403e6674291b124d2c69af56e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:31 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:45988
I20260806 12:59:31.042686 8685 tablet_replica.cc:330] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: stopping tablet replica
I20260806 12:59:31.042793 8685 raft_consensus.cc:2229] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:59:31.042886 8685 raft_consensus.cc:2258] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:31.043807 8685 ts_tablet_manager.cc:1905] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:59:31.044626 8577 tablet_service.cc:1514] Processing DeleteTablet for tablet fcac4403e6674291b124d2c69af56e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:31 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:34296
I20260806 12:59:31.045877 8685 ts_tablet_manager.cc:1918] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:31.045957 8685 log.cc:1196] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/fcac4403e6674291b124d2c69af56e0a
I20260806 12:59:31.046286 8685 ts_tablet_manager.cc:1939] T fcac4403e6674291b124d2c69af56e0a P 9638e09612a944cab7e51f95a2016037: Deleting consensus metadata
I20260806 12:59:31.046692 8160 catalog_manager.cc:4841] TS 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305): tablet fcac4403e6674291b124d2c69af56e0a (table test-table [id=3911ae981cb543409960e51bf2fc5259]) successfully deleted
I20260806 12:59:31.046861 8686 tablet_replica.cc:330] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1: stopping tablet replica
I20260806 12:59:31.046964 8686 raft_consensus.cc:2229] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:59:31.047071 8686 raft_consensus.cc:2258] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:31.047490 8686 ts_tablet_manager.cc:1905] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 12:59:31.049486 8163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:46053 (error 108)
I20260806 12:59:31.049507 8686 ts_tablet_manager.cc:1918] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:31.049571 8686 log.cc:1196] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/fcac4403e6674291b124d2c69af56e0a
I20260806 12:59:31.049782 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 8377
I20260806 12:59:31.049844 8686 ts_tablet_manager.cc:1939] T fcac4403e6674291b124d2c69af56e0a P 6e68537becb04c1c99d4981ffa922bd1: Deleting consensus metadata
I20260806 12:59:31.050374 8657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:31.049966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:46053 (local address 127.0.0.1:40506)
0806 12:59:31.050297 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:31.050334 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309}
W20260806 12:59:31.050778 8163 catalog_manager.cc:4584] TS 7fff71f6435f4c2c91ea52bc0931259f (127.24.83.1:46053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fcac4403e6674291b124d2c69af56e0a: Network error: Client connection negotiation failed: client connection to 127.24.83.1:46053: connect: Connection refused (error 111)
I20260806 12:59:31.051908 8160 catalog_manager.cc:4841] TS 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223): tablet fcac4403e6674291b124d2c69af56e0a (table test-table [id=3911ae981cb543409960e51bf2fc5259]) successfully deleted
I20260806 12:59:31.055347 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 8513
I20260806 12:59:31.060761 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 8143
I20260806 12:59:51.066355 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:34107
--webserver_interface=127.24.83.62
--webserver_port=41417
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:34107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:59:51.147650 8688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:51.147879 8688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:51.147964 8688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:51.149098 8688 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 12:59:51.149135 8688 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 12:59:51.149148 8688 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 12:59:51.149160 8688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:59:51.149173 8688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:51.149186 8688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:51.149206 8688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:59:51.149219 8688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:59:51.150635 8688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:34107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:34107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=41417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:51.150899 8688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:51.151100 8688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:51.159996 8694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:51.160238 8688 server_base.cc:1034] running on GCE node
W20260806 12:59:51.159999 8695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:51.159996 8697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:51.160634 8688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:51.161026 8688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:51.162199 8688 hybrid_clock.cc:648] HybridClock initialized: now 1786021191162175 us; error 37 us; skew 500 ppm
Aug 06 12:59:51 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:51.164893 8688 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 12:59:51.165365 8688 webserver.cc:460] Webserver started at http://127.24.83.62:41417/ using document root <none> and password file <none>
I20260806 12:59:51.165623 8688 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:51.165709 8688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:51.167109 8688 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 12:59:51.167810 8704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:51.168017 8688 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:51.168128 8688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "049b7e608ae54aefafae8104c741fec1"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "8ede9399ba21bd54da1527d08362fa7d"
server_key_iv: "533e38db3210dc18c2cfa0eecd35f211"
server_key_version: "encryptionkey@0"
I20260806 12:59:51.168534 8688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:51.180148 8688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:51.182348 8688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:51.182502 8688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:51.186669 8688 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:34107
I20260806 12:59:51.186702 8758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:34107 every 8 connection(s)
I20260806 12:59:51.187109 8688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 12:59:51.188762 8759 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:59:51.189389 8759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Bootstrap starting.
I20260806 12:59:51.191457 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 8688
I20260806 12:59:51.191761 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:46053
--local_ip_for_outbound_sockets=127.24.83.1
--tserver_master_addrs=127.24.83.62:34107
--webserver_port=41183
--webserver_interface=127.24.83.1
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:51.199033 8759 log.cc:824] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:51.201130 8759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:59:51.201457 8759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Bootstrap complete.
I20260806 12:59:51.204069 8759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 12:59:51.204335 8759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049b7e608ae54aefafae8104c741fec1, State: Initialized, Role: FOLLOWER
I20260806 12:59:51.204548 8759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 12:59:51.204645 8759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:59:51.204710 8759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:59:51.204774 8759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:59:51.205888 8759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 12:59:51.206017 8759 leader_election.cc:302] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 049b7e608ae54aefafae8104c741fec1; no voters:
I20260806 12:59:51.206233 8759 leader_election.cc:288] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 12:59:51.206279 8763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 12:59:51.206471 8763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 2 LEADER]: Becoming Leader. State: Replica: 049b7e608ae54aefafae8104c741fec1, State: Running, Role: LEADER
I20260806 12:59:51.206528 8759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:59:51.207718 8763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 12:59:51.208420 8765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 049b7e608ae54aefafae8104c741fec1. Latest consensus state: current_term: 2 leader_uuid: "049b7e608ae54aefafae8104c741fec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } } }
I20260806 12:59:51.208544 8765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:51.208906 8764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "049b7e608ae54aefafae8104c741fec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } } }
I20260806 12:59:51.208979 8764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: This master's current role is: LEADER
I20260806 12:59:51.209060 8778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:59:51.209905 8778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:59:51.210019 8778 catalog_manager.cc:1259] Loaded cluster ID: 0a48e20e0d6d4e74afedac5fb1eec1ac
I20260806 12:59:51.210055 8778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:59:51.211359 8778 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:59:51.211562 8778 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Loaded TSK: 0
I20260806 12:59:51.221753 8778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Generated new TSK 1
I20260806 12:59:51.221882 8778 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 12:59:51.276760 8761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:51.277010 8761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:51.277050 8761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:51.278208 8761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:51.278273 8761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:51.278319 8761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 12:59:51.279791 8761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:46053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=41183
--tserver_master_addrs=127.24.83.62:34107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:51.280083 8761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:51.280319 8761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:51.286166 8789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:51.286412 8761 server_base.cc:1034] running on GCE node
W20260806 12:59:51.286154 8786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:51.286163 8787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:51.286736 8761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:51.287024 8761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:51.288175 8761 hybrid_clock.cc:648] HybridClock initialized: now 1786021191288155 us; error 36 us; skew 500 ppm
Aug 06 12:59:51 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:51.290696 8761 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 12:59:51.291199 8761 webserver.cc:460] Webserver started at http://127.24.83.1:41183/ using document root <none> and password file <none>
I20260806 12:59:51.291433 8761 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:51.291489 8761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:51.292977 8761 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:59:51.294854 8796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:51.295135 8761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:59:51.295235 8761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "7fff71f6435f4c2c91ea52bc0931259f"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "371a1815ee82edfb78a5353e942181f0"
server_key_iv: "d047f18e8bd1a5aa8163b7ee98a225ff"
server_key_version: "encryptionkey@0"
I20260806 12:59:51.295693 8761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:51.310840 8761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:51.313218 8761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:51.313374 8761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:51.313621 8761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:51.314098 8803 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 12:59:51.316073 8761 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 12:59:51.316145 8761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 12:59:51.316191 8761 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 12:59:51.316846 8761 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 12:59:51.316910 8761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 12:59:51.316967 8803 tablet_bootstrap.cc:492] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Bootstrap starting.
I20260806 12:59:51.324208 8761 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:46053
I20260806 12:59:51.324579 8761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 12:59:51.326865 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 8761
I20260806 12:59:51.327172 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:45305
--local_ip_for_outbound_sockets=127.24.83.2
--tserver_master_addrs=127.24.83.62:34107
--webserver_port=41027
--webserver_interface=127.24.83.2
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:51.328078 8912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:46053 every 8 connection(s)
I20260806 12:59:51.328670 8803 log.cc:824] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:51.331990 8803 tablet_bootstrap.cc:492] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:59:51.332304 8803 tablet_bootstrap.cc:492] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Bootstrap complete.
I20260806 12:59:51.333619 8803 ts_tablet_manager.cc:1397] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.013s
I20260806 12:59:51.335107 8803 raft_consensus.cc:348] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:51.335522 8803 raft_consensus.cc:729] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fff71f6435f4c2c91ea52bc0931259f, State: Initialized, Role: FOLLOWER
I20260806 12:59:51.335739 8803 consensus_queue.cc:260] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:51.336118 8803 ts_tablet_manager.cc:1428] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Aug 06 12:59:51 dist-test-slave-zkpd krb5kdc[8130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:51.344835 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.325414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:60903 (local address 127.24.83.62:34107)
0806 12:59:51.332367 (+ 6953us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.332377 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.332417 (+ 40us) server_negotiation.cc:407] Connection header received
0806 12:59:51.332550 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.332557 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.332612 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.332747 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:51.333825 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.335411 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.337110 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.337725 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.337852 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.340701 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:51.340705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:51.340705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:51.340719 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:51.342895 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.343395 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.343398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.343398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.343508 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.343768 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.343769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.343769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.344072 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:51.344152 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:51.344483 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 12:59:51.344617 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6893,"thread_start_us":85,"threads_started":1}
I20260806 12:59:51.344838 8916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.325307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:60903)
0806 12:59:51.328990 (+ 3683us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:51.329036 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:51.329665 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:51.332937 (+ 3272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:51.332941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:51.333363 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:51.333658 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.333666 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.335552 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.335553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.336994 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.336997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.337838 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.337839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.338002 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.338676 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:51.338687 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:51.340564 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:51.342992 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:51.342999 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:51.343001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:51.343298 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:51.343579 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:51.343580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:51.343580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:51.343708 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:51.344141 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:51.344148 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:51.344300 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 12:59:51.344396 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 12:59:51.344547 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3638,"thread_start_us":93,"threads_started":1}
I20260806 12:59:51.345935 8913 heartbeater.cc:347] Connected to a master server at 127.24.83.62:34107
I20260806 12:59:51.346022 8913 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:51.346184 8913 heartbeater.cc:511] Master 127.24.83.62:34107 requested a full tablet report, sending...
I20260806 12:59:51.346992 8722 ts_manager.cc:194] Registered new tserver with Master: 7fff71f6435f4c2c91ea52bc0931259f (127.24.83.1:46053)
I20260806 12:59:51.348765 8722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:60903
I20260806 12:59:51.349746 8913 heartbeater.cc:503] Master 127.24.83.62:34107 was elected leader, sending a full tablet report...
Aug 06 12:59:51 dist-test-slave-zkpd krb5kdc[8130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.1@KRBTEST.COM
I20260806 12:59:51.357851 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.348076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:46053 (local address 127.0.0.1:56458)
0806 12:59:51.348270 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:51.348314 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:51.348410 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:51.348698 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:51.348702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:51.349026 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:51.349534 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.349536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.350789 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.350790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.351769 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.351772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.352546 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.352547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.352633 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.353110 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:51.353115 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:51.354675 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:51.356627 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:51.356631 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:51.356633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:51.356923 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:51.357137 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:51.357137 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:51.357138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:51.357201 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:51.357484 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:51.357490 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:51.357537 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 12:59:51.357636 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 12:59:51.357745 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260806 12:59:51.357853 8922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.348348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.24.83.1:46053)
0806 12:59:51.348553 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.348557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.348566 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:59:51.348596 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.348598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.348613 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.349622 (+ 1009us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:51.349674 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.350699 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.351868 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.352453 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.352537 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.354767 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:51.354775 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:51.354775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:51.354782 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:51.356539 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.357006 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.357009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.357009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.357069 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.357264 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.357264 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.357265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.357408 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:51.357499 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:51.357690 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 12:59:51.357769 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":86,"threads_started":1}
I20260806 12:59:51.358878 8845 tablet_service.cc:1514] Processing DeleteTablet for tablet fcac4403e6674291b124d2c69af56e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:31 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56458
I20260806 12:59:51.359474 8923 tablet_replica.cc:330] stopping tablet replica
I20260806 12:59:51.359622 8923 raft_consensus.cc:2229] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:59:51.359731 8923 raft_consensus.cc:2258] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:51.360177 8923 ts_tablet_manager.cc:1905] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:59:51.362754 8923 ts_tablet_manager.cc:1918] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:51.362833 8923 log.cc:1196] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal/wals/fcac4403e6674291b124d2c69af56e0a
I20260806 12:59:51.363106 8923 ts_tablet_manager.cc:1939] T fcac4403e6674291b124d2c69af56e0a P 7fff71f6435f4c2c91ea52bc0931259f: Deleting consensus metadata
I20260806 12:59:51.363585 8708 catalog_manager.cc:4841] TS 7fff71f6435f4c2c91ea52bc0931259f (127.24.83.1:46053): tablet fcac4403e6674291b124d2c69af56e0a (table test-table [id=3911ae981cb543409960e51bf2fc5259]) successfully deleted
W20260806 12:59:51.420313 8918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:51.420528 8918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:51.420570 8918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:51.421710 8918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:51.421772 8918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:51.421818 8918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 12:59:51.423243 8918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:45305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=41027
--tserver_master_addrs=127.24.83.62:34107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:51.423508 8918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:51.423719 8918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:59:51.428592 8918 server_base.cc:1034] running on GCE node
W20260806 12:59:51.428540 8928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:51.428547 8929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:51.428570 8931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:51.429010 8918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:51.429298 8918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:51.430470 8918 hybrid_clock.cc:648] HybridClock initialized: now 1786021191430458 us; error 42 us; skew 500 ppm
Aug 06 12:59:51 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:51.432875 8918 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 12:59:51.433327 8918 webserver.cc:460] Webserver started at http://127.24.83.2:41027/ using document root <none> and password file <none>
I20260806 12:59:51.433550 8918 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:51.433630 8918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:51.434952 8918 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:51.435530 8938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:51.435689 8918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 12:59:51.435784 8918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "9638e09612a944cab7e51f95a2016037"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "dd04611faec26138ef66d56a111c69e0"
server_key_iv: "ac03737a1c36690f02429f7b8caf5f4f"
server_key_version: "encryptionkey@0"
I20260806 12:59:51.436152 8918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:51.446863 8918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:51.448647 8918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:51.448798 8918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:51.449008 8918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:51.449299 8918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:51.449349 8918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:51.449399 8918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:51.449440 8918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:51.455106 8918 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:45305
I20260806 12:59:51.455140 9054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:45305 every 8 connection(s)
I20260806 12:59:51.455463 8918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:59:51 dist-test-slave-zkpd krb5kdc[8130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:51.465359 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 8918
I20260806 12:59:51.465672 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:43223
--local_ip_for_outbound_sockets=127.24.83.3
--tserver_master_addrs=127.24.83.62:34107
--webserver_port=36225
--webserver_interface=127.24.83.3
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:59:51.470849 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.456214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:35055 (local address 127.24.83.62:34107)
0806 12:59:51.456288 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.456290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.456508 (+ 218us) server_negotiation.cc:407] Connection header received
0806 12:59:51.456994 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.456995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.457012 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.457076 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:51.457966 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.458701 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.460298 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.460630 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.460679 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.465633 (+ 4954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:51.465634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:51.465635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:51.465642 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:51.469046 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.469557 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.469559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.469559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.469618 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.469928 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.469929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.469930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.470083 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:51.470134 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:51.470607 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 12:59:51.470713 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:59:51.471699 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.456215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:35055)
0806 12:59:51.456390 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:51.456425 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:51.456894 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:51.457147 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:51.457151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:51.457688 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:51.457864 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.457869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.458848 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.458849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.460190 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.460193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.460788 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.460789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.460938 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.461509 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:51.461523 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:51.465547 (+ 4024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:51.469139 (+ 3592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:51.469146 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:51.469148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:51.469469 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:51.469687 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:51.469688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:51.469688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:51.469834 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:51.470189 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:51.470198 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:51.470435 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 12:59:51.471206 (+ 771us) client_negotiation.cc:240] Negotiation successful
0806 12:59:51.471415 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260806 12:59:51.472286 9055 heartbeater.cc:347] Connected to a master server at 127.24.83.62:34107
I20260806 12:59:51.472395 9055 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:51.472604 9055 heartbeater.cc:511] Master 127.24.83.62:34107 requested a full tablet report, sending...
I20260806 12:59:51.472970 8722 ts_manager.cc:194] Registered new tserver with Master: 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305)
I20260806 12:59:51.473345 8722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:35055
W20260806 12:59:51.549203 9058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:59:51.549394 9058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:59:51.549417 9058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:59:51.550547 9058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:59:51.550591 9058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:59:51.550617 9058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 12:59:51.552091 9058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:43223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=36225
--tserver_master_addrs=127.24.83.62:34107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 12:59:51.552285 9058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:59:51.552536 9058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:59:51.557426 9064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:51.557425 9067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:59:51.557515 9063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:59:51.557605 9058 server_base.cc:1034] running on GCE node
I20260806 12:59:51.557835 9058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:59:51.558081 9058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:59:51.559238 9058 hybrid_clock.cc:648] HybridClock initialized: now 1786021191559226 us; error 28 us; skew 500 ppm
Aug 06 12:59:51 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:59:51.561678 9058 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 12:59:51.562083 9058 webserver.cc:460] Webserver started at http://127.24.83.3:36225/ using document root <none> and password file <none>
I20260806 12:59:51.562305 9058 fs_manager.cc:359] Metadata directory not provided
I20260806 12:59:51.562392 9058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:59:51.563647 9058 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:51.564246 9074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:51.564421 9058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:51.564518 9058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "6e68537becb04c1c99d4981ffa922bd1"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "ad87e76e9eef9bc4e8458cf33d498ac9"
server_key_iv: "c26e62653aafbcb1084b55e650cbccdc"
server_key_version: "encryptionkey@0"
I20260806 12:59:51.564870 9058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:59:51.577474 9058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:59:51.579937 9058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:59:51.580099 9058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:59:51.580319 9058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:59:51.580607 9058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:59:51.580641 9058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:51.580662 9058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:59:51.580706 9058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:59:51.586232 9058 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:43223
I20260806 12:59:51.586311 9190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:43223 every 8 connection(s)
I20260806 12:59:51.586652 9058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 12:59:51.591814 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 9058
Aug 06 12:59:51 dist-test-slave-zkpd krb5kdc[8130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 12:59:51.596134 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.587363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:33919 (local address 127.24.83.62:34107)
0806 12:59:51.587425 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.587427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.587626 (+ 199us) server_negotiation.cc:407] Connection header received
0806 12:59:51.588200 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.588202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.588216 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.588279 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:51.589136 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.589622 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.590614 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.590817 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.590905 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.593102 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:51.593104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:51.593104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:51.593111 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:51.594663 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.595065 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.595066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.595066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.595116 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.595386 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.595387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.595387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.595510 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:51.595588 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:51.595946 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 12:59:51.596031 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 12:59:51.596230 9193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.587356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:33919)
0806 12:59:51.587512 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:51.587545 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:51.588098 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:51.588341 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:51.588344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:51.588855 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:59:51.589025 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.589030 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.589732 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.589733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.590518 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.590520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.590916 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.590917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.591042 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.591527 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:59:51.591534 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:59:51.593005 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:59:51.594746 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:51.594753 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:51.594754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:51.594971 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:51.595176 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:59:51.595176 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:59:51.595177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:59:51.595299 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:59:51.595583 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:59:51.595589 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:59:51.595769 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 12:59:51.595881 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 12:59:51.596048 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260806 12:59:51.596724 9191 heartbeater.cc:347] Connected to a master server at 127.24.83.62:34107
I20260806 12:59:51.596829 9191 heartbeater.cc:464] Registering TS with master...
I20260806 12:59:51.597009 9191 heartbeater.cc:511] Master 127.24.83.62:34107 requested a full tablet report, sending...
I20260806 12:59:51.597396 8722 ts_manager.cc:194] Registered new tserver with Master: 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223)
I20260806 12:59:51.597906 8722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:33919
I20260806 12:59:51.599606 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.592272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59022 (local address 127.24.83.62:34107)
0806 12:59:51.592407 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.592409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.592477 (+ 68us) server_negotiation.cc:407] Connection header received
0806 12:59:51.592548 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.592549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.592561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.592641 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:51.593242 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.593943 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.595045 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.595319 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.595381 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.596844 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:51.596846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:51.596846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:51.596852 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:51.598467 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.598803 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.598804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.598804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.598859 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.599076 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.599077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.599077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.599200 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:51.599383 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:51.599433 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 12:59:51.599508 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":54,"threads_started":1}
I20260806 12:59:51.600366 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:59:51.602787 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.600669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59026 (local address 127.24.83.62:34107)
0806 12:59:51.600733 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.600735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.600780 (+ 45us) server_negotiation.cc:407] Connection header received
0806 12:59:51.600839 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.600840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.600864 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.600953 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:59:51.601028 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.601480 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.602157 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.602370 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.602434 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.602555 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:59:51.602590 (+ 35us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 12:59:51.602709 (+ 119us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59026: token expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 12:59:51.602845 9195 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59026: token expired
W20260806 12:59:51.603003 8650 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.83.62:34107)
I20260806 12:59:51.608510 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.603196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59034 (local address 127.24.83.62:34107)
0806 12:59:51.603262 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.603264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.603284 (+ 20us) server_negotiation.cc:407] Connection header received
0806 12:59:51.603331 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.603331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.603338 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.603377 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:59:51.603789 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.604292 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.604910 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.605074 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.605122 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.606110 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:59:51.606111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:59:51.606111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:59:51.606117 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:59:51.607439 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.607794 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.607795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.607795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.607850 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:59:51.608059 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:59:51.608060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:59:51.608060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:59:51.608151 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:59:51.608226 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:51.608325 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 12:59:51.608384 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:59:51.609486 8722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:59:51.610980 8722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:59:51.613006 8845 tablet_service.cc:1467] Processing CreateTablet for tablet eda53d7ce8544d05a8220354f97ccf94 (DEFAULT_TABLE table=test-table [id=6b87a72dacd246108a55aea316bfee85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:51.613215 8845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda53d7ce8544d05a8220354f97ccf94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:51.614799 8803 tablet_bootstrap.cc:492] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: Bootstrap starting.
I20260806 12:59:51.615147 8803 tablet_bootstrap.cc:654] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:51.615739 8803 tablet_bootstrap.cc:492] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: No bootstrap required, opened a new log
I20260806 12:59:51.615787 8803 ts_tablet_manager.cc:1397] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:51.616143 8803 raft_consensus.cc:348] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:51.616216 8803 raft_consensus.cc:374] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:51.616257 8803 raft_consensus.cc:729] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fff71f6435f4c2c91ea52bc0931259f, State: Initialized, Role: FOLLOWER
I20260806 12:59:51.616312 8803 consensus_queue.cc:260] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:51.616434 8803 ts_tablet_manager.cc:1428] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:59:51.617025 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.612765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:45305 (local address 127.0.0.1:39486)
0806 12:59:51.612800 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:51.612811 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:51.612845 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:51.613378 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:51.613379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:51.613381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:51.613428 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.613429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.614605 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.614606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.615650 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.615653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.616754 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.616755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.616837 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.616855 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 12:59:51.616881 (+ 26us) client_negotiation.cc:240] Negotiation successful
0806 12:59:51.616952 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 12:59:51.617388 9197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.612865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39486 (local address 127.24.83.2:45305)
0806 12:59:51.613216 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.613220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.613231 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:59:51.613272 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.613274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.613291 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.613372 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:51.613477 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.614521 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.615973 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.616624 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.617162 (+ 538us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.617249 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:51.617277 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 12:59:51.617326 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":98,"threads_started":1}
I20260806 12:59:51.618046 8987 tablet_service.cc:1467] Processing CreateTablet for tablet eda53d7ce8544d05a8220354f97ccf94 (DEFAULT_TABLE table=test-table [id=6b87a72dacd246108a55aea316bfee85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:51.618299 9198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.612857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:43223 (local address 127.0.0.1:34206)
0806 12:59:51.613031 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:51.613042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:51.613086 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:51.615045 (+ 1959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:51.615045 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:51.615047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:51.615082 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.615083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.616357 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.616358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.617389 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.617392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.618065 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.618066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.618132 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.618136 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:59:51.618194 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 12:59:51.618232 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
I20260806 12:59:51.618312 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.613062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34206 (local address 127.24.83.3:43223)
0806 12:59:51.614877 (+ 1815us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.614881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.614900 (+ 19us) server_negotiation.cc:407] Connection header received
0806 12:59:51.614943 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.614947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.614963 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.615049 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:51.615137 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.616252 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.617486 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.617977 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.618064 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.618116 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:51.618206 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 12:59:51.618241 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1783,"thread_start_us":70,"threads_started":1}
I20260806 12:59:51.618474 8987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda53d7ce8544d05a8220354f97ccf94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:51.618957 9124 tablet_service.cc:1467] Processing CreateTablet for tablet eda53d7ce8544d05a8220354f97ccf94 (DEFAULT_TABLE table=test-table [id=6b87a72dacd246108a55aea316bfee85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:59:51.619180 9124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda53d7ce8544d05a8220354f97ccf94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:59:51.621491 9201 tablet_bootstrap.cc:492] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: Bootstrap starting.
I20260806 12:59:51.621973 9202 tablet_bootstrap.cc:492] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1: Bootstrap starting.
I20260806 12:59:51.622148 9201 tablet_bootstrap.cc:654] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:51.622557 9202 tablet_bootstrap.cc:654] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1: Neither blocks nor log segments found. Creating new log.
I20260806 12:59:51.622658 9201 log.cc:824] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:51.623101 9202 log.cc:824] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1: Log is configured to *not* fsync() on all Append() calls
I20260806 12:59:51.623440 9201 tablet_bootstrap.cc:492] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: No bootstrap required, opened a new log
I20260806 12:59:51.623517 9201 ts_tablet_manager.cc:1397] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:59:51.623836 9202 tablet_bootstrap.cc:492] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1: No bootstrap required, opened a new log
I20260806 12:59:51.623947 9202 ts_tablet_manager.cc:1397] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:59:51.625159 9201 raft_consensus.cc:348] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:51.625340 9201 raft_consensus.cc:374] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:51.625365 9201 raft_consensus.cc:729] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9638e09612a944cab7e51f95a2016037, State: Initialized, Role: FOLLOWER
I20260806 12:59:51.625497 9201 consensus_queue.cc:260] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:51.625820 9201 ts_tablet_manager.cc:1428] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 12:59:51.625854 9055 heartbeater.cc:503] Master 127.24.83.62:34107 was elected leader, sending a full tablet report...
I20260806 12:59:51.625730 9202 raft_consensus.cc:348] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:51.625908 9202 raft_consensus.cc:374] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:59:51.625952 9202 raft_consensus.cc:729] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e68537becb04c1c99d4981ffa922bd1, State: Initialized, Role: FOLLOWER
I20260806 12:59:51.626072 9202 consensus_queue.cc:260] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:51.626394 9202 ts_tablet_manager.cc:1428] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:59:51.626503 9191 heartbeater.cc:503] Master 127.24.83.62:34107 was elected leader, sending a full tablet report...
W20260806 12:59:51.706030 9056 tablet.cc:2369] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:59:51.707472 8920 raft_consensus.cc:482] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:59:51.707599 8920 raft_consensus.cc:504] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:51.708009 8920 leader_election.cc:288] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305), 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223)
I20260806 12:59:51.710896 9197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.708302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:50587 (local address 127.24.83.2:45305)
0806 12:59:51.708349 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.708352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.708374 (+ 22us) server_negotiation.cc:407] Connection header received
0806 12:59:51.708412 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.708413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.708424 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.708473 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:51.708671 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.709166 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.710115 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.710515 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.710598 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.710609 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:51.710794 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 12:59:51.710837 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 12:59:51.710919 8916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.708183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:45305 (local address 127.24.83.1:50587)
0806 12:59:51.708265 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:51.708278 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:51.708339 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:51.708523 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:51.708525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:51.708526 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:51.708612 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.708613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.709253 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.709254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.710011 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.710013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.710599 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.710600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.710688 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.710707 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 12:59:51.710774 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 12:59:51.710819 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 12:59:51.711197 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.708334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:43223 (local address 127.24.83.1:52973)
0806 12:59:51.708499 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:51.708510 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:59:51.708562 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:59:51.708687 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:59:51.708688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:59:51.708689 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:59:51.708763 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.708765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.709579 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.709580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.710318 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:59:51.710320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.710898 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.710899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:59:51.711000 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.711005 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:59:51.711071 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 12:59:51.711118 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":117,"threads_started":1}
I20260806 12:59:51.711205 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.708539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:52973 (local address 127.24.83.3:43223)
0806 12:59:51.708583 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.708585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.708594 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:59:51.708618 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.708619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.708629 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.708683 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:59:51.708846 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.709494 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.710411 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.710796 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.710881 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.710891 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:51.711078 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 12:59:51.711126 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 12:59:51.711254 9008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda53d7ce8544d05a8220354f97ccf94" candidate_uuid: "7fff71f6435f4c2c91ea52bc0931259f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9638e09612a944cab7e51f95a2016037" is_pre_election: true
I20260806 12:59:51.711390 9008 raft_consensus.cc:2454] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fff71f6435f4c2c91ea52bc0931259f in term 0.
I20260806 12:59:51.711613 8797 leader_election.cc:302] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fff71f6435f4c2c91ea52bc0931259f, 9638e09612a944cab7e51f95a2016037; no voters:
I20260806 12:59:51.711709 9144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda53d7ce8544d05a8220354f97ccf94" candidate_uuid: "7fff71f6435f4c2c91ea52bc0931259f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68537becb04c1c99d4981ffa922bd1" is_pre_election: true
I20260806 12:59:51.711766 8920 raft_consensus.cc:2781] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:59:51.711858 9144 raft_consensus.cc:2454] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fff71f6435f4c2c91ea52bc0931259f in term 0.
I20260806 12:59:51.711872 8920 raft_consensus.cc:482] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:59:51.711932 8920 raft_consensus.cc:3037] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:51.712688 8920 raft_consensus.cc:504] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:51.712836 8920 leader_election.cc:288] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 1 election: Requested vote from peers 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305), 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223)
I20260806 12:59:51.712983 9008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda53d7ce8544d05a8220354f97ccf94" candidate_uuid: "7fff71f6435f4c2c91ea52bc0931259f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9638e09612a944cab7e51f95a2016037"
I20260806 12:59:51.713014 9144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda53d7ce8544d05a8220354f97ccf94" candidate_uuid: "7fff71f6435f4c2c91ea52bc0931259f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e68537becb04c1c99d4981ffa922bd1"
I20260806 12:59:51.713083 9008 raft_consensus.cc:3037] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:51.713090 9144 raft_consensus.cc:3037] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:59:51.714003 9144 raft_consensus.cc:2454] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fff71f6435f4c2c91ea52bc0931259f in term 1.
I20260806 12:59:51.714003 9008 raft_consensus.cc:2454] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fff71f6435f4c2c91ea52bc0931259f in term 1.
I20260806 12:59:51.714170 8797 leader_election.cc:302] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e68537becb04c1c99d4981ffa922bd1, 7fff71f6435f4c2c91ea52bc0931259f; no voters:
I20260806 12:59:51.714272 8920 raft_consensus.cc:2781] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:59:51.714423 8920 raft_consensus.cc:686] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 LEADER]: Becoming Leader. State: Replica: 7fff71f6435f4c2c91ea52bc0931259f, State: Running, Role: LEADER
I20260806 12:59:51.714488 8920 consensus_queue.cc:237] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 12:59:51.715242 8722 catalog_manager.cc:5495] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fff71f6435f4c2c91ea52bc0931259f (127.24.83.1). New cstate: current_term: 1 leader_uuid: "7fff71f6435f4c2c91ea52bc0931259f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } health_report { overall_health: HEALTHY } } }
I20260806 12:59:51.731765 8922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.729430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56462 (local address 127.24.83.1:46053)
0806 12:59:51.729491 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 12:59:51.729494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:59:51.729503 (+ 9us) server_negotiation.cc:407] Connection header received
0806 12:59:51.729586 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:59:51.729587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:59:51.729604 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:59:51.729673 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:59:51.729817 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.730280 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.730984 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:59:51.731210 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:59:51.731287 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:59:51.731440 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:59:51.731542 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:59:51.731625 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 12:59:51.731666 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 12:59:51.731687 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:59:51.733210 9144 raft_consensus.cc:1261] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1 [term 1 FOLLOWER]: Refusing update from remote peer 7fff71f6435f4c2c91ea52bc0931259f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:51.733210 9008 raft_consensus.cc:1261] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037 [term 1 FOLLOWER]: Refusing update from remote peer 7fff71f6435f4c2c91ea52bc0931259f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:59:51.733558 9209 consensus_queue.cc:1035] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:51.733809 8920 consensus_queue.cc:1035] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:59:51.736243 9214 mvcc.cc:205] Tried to move back new op lower bound from 7315542801336606720 to 7315542801263128576. Current Snapshot: MvccSnapshot[applied={T|T < 7315542801336606720}]
I20260806 12:59:51.738238 8721 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:59:51.738332 8721 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:59:51.739094 8721 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Sending DeleteTablet for 3 replicas of tablet eda53d7ce8544d05a8220354f97ccf94
I20260806 12:59:51.739389 8845 tablet_service.cc:1514] Processing DeleteTablet for tablet eda53d7ce8544d05a8220354f97ccf94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:51 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:56458
I20260806 12:59:51.739440 9124 tablet_service.cc:1514] Processing DeleteTablet for tablet eda53d7ce8544d05a8220354f97ccf94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:51 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:34206
I20260806 12:59:51.739480 8923 tablet_replica.cc:330] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: stopping tablet replica
I20260806 12:59:51.739466 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 8761
I20260806 12:59:51.739526 8923 raft_consensus.cc:2229] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:59:51.739614 8923 raft_consensus.cc:2258] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:51.742452 8987 tablet_service.cc:1514] Processing DeleteTablet for tablet eda53d7ce8544d05a8220354f97ccf94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:59:51 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:39486
I20260806 12:59:51.742702 9222 tablet_replica.cc:330] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1: stopping tablet replica
I20260806 12:59:51.742697 9223 tablet_replica.cc:330] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: stopping tablet replica
I20260806 12:59:51.742816 9223 raft_consensus.cc:2229] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:59:51.742823 9222 raft_consensus.cc:2229] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:59:51.742925 9222 raft_consensus.cc:2258] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:51.742933 9223 raft_consensus.cc:2258] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:59:51.743387 9222 ts_tablet_manager.cc:1905] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:59:51.743418 9223 ts_tablet_manager.cc:1905] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:59:51.745180 9223 ts_tablet_manager.cc:1918] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:51.745285 9223 log.cc:1196] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/eda53d7ce8544d05a8220354f97ccf94
I20260806 12:59:51.745371 9222 ts_tablet_manager.cc:1918] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:59:51.745450 9222 log.cc:1196] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/eda53d7ce8544d05a8220354f97ccf94
I20260806 12:59:51.745627 9223 ts_tablet_manager.cc:1939] T eda53d7ce8544d05a8220354f97ccf94 P 9638e09612a944cab7e51f95a2016037: Deleting consensus metadata
I20260806 12:59:51.745726 9222 ts_tablet_manager.cc:1939] T eda53d7ce8544d05a8220354f97ccf94 P 6e68537becb04c1c99d4981ffa922bd1: Deleting consensus metadata
I20260806 12:59:51.746069 8705 catalog_manager.cc:4841] TS 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305): tablet eda53d7ce8544d05a8220354f97ccf94 (table test-table [id=6b87a72dacd246108a55aea316bfee85]) successfully deleted
I20260806 12:59:51.746268 8705 catalog_manager.cc:4841] TS 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223): tablet eda53d7ce8544d05a8220354f97ccf94 (table test-table [id=6b87a72dacd246108a55aea316bfee85]) successfully deleted
W20260806 12:59:51.747326 8708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:46053 (error 108)
I20260806 12:59:51.747805 9198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:59:51.747678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:46053 (local address 127.0.0.1:56478)
0806 12:59:51.747747 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 12:59:51.747778 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 12:59:51.747983 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 8918
W20260806 12:59:51.747952 8708 catalog_manager.cc:4584] TS 7fff71f6435f4c2c91ea52bc0931259f (127.24.83.1:46053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eda53d7ce8544d05a8220354f97ccf94: Network error: Client connection negotiation failed: client connection to 127.24.83.1:46053: connect: Connection refused (error 111)
I20260806 12:59:51.755824 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 9058
I20260806 12:59:51.760658 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 8688
I20260806 13:00:11.766530 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:34107
--webserver_interface=127.24.83.62
--webserver_port=41417
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:34107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:11.849144 9225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:11.849388 9225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:11.849429 9225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:11.850580 9225 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 13:00:11.850641 9225 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 13:00:11.850663 9225 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 13:00:11.850694 9225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:11.850723 9225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:11.850754 9225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:11.850780 9225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:11.850803 9225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:11.852366 9225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:34107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:34107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=41417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:11.852641 9225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:11.852866 9225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:11.858026 9233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:11.858142 9230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:11.858055 9225 server_base.cc:1034] running on GCE node
W20260806 13:00:11.858021 9231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:11.858392 9225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:11.858680 9225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:11.859853 9225 hybrid_clock.cc:648] HybridClock initialized: now 1786021211859844 us; error 36 us; skew 500 ppm
Aug 06 13:00:11 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021211, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:11.862365 9225 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:00:11.862738 9225 webserver.cc:460] Webserver started at http://127.24.83.62:41417/ using document root <none> and password file <none>
I20260806 13:00:11.862967 9225 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:11.863015 9225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:11.864449 9225 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:11.865263 9240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:11.865473 9225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:11.865574 9225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "049b7e608ae54aefafae8104c741fec1"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "8ede9399ba21bd54da1527d08362fa7d"
server_key_iv: "533e38db3210dc18c2cfa0eecd35f211"
server_key_version: "encryptionkey@0"
I20260806 13:00:11.865917 9225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:11.885756 9225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:11.888218 9225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:11.888389 9225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:11.892798 9225 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:34107
I20260806 13:00:11.892838 9292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:34107 every 8 connection(s)
I20260806 13:00:11.893239 9225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:11.894721 9293 sys_catalog.cc:263] Verifying existing consensus state
I20260806 13:00:11.895409 9293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Bootstrap starting.
I20260806 13:00:11.901974 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 9225
I20260806 13:00:11.902274 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:46053
--local_ip_for_outbound_sockets=127.24.83.1
--tserver_master_addrs=127.24.83.62:34107
--webserver_port=41183
--webserver_interface=127.24.83.1
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:00:11.905079 9293 log.cc:824] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:11.907456 9293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:00:11.907778 9293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Bootstrap complete.
I20260806 13:00:11.910153 9293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 13:00:11.910423 9293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049b7e608ae54aefafae8104c741fec1, State: Initialized, Role: FOLLOWER
I20260806 13:00:11.910622 9293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 13:00:11.910718 9293 raft_consensus.cc:388] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:11.910773 9293 raft_consensus.cc:482] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:11.910822 9293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 2 FOLLOWER]: Advancing to term 3
I20260806 13:00:11.911839 9293 raft_consensus.cc:504] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 13:00:11.911978 9293 leader_election.cc:302] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 049b7e608ae54aefafae8104c741fec1; no voters:
I20260806 13:00:11.912174 9293 leader_election.cc:288] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 13:00:11.912215 9298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 13:00:11.912336 9298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [term 3 LEADER]: Becoming Leader. State: Replica: 049b7e608ae54aefafae8104c741fec1, State: Running, Role: LEADER
I20260806 13:00:11.912472 9293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:11.912437 9298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } }
I20260806 13:00:11.912765 9298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 049b7e608ae54aefafae8104c741fec1. Latest consensus state: current_term: 3 leader_uuid: "049b7e608ae54aefafae8104c741fec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } } }
I20260806 13:00:11.912837 9298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:11.912994 9299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "049b7e608ae54aefafae8104c741fec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049b7e608ae54aefafae8104c741fec1" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 34107 } } }
I20260806 13:00:11.913136 9299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:11.913072 9303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:11.914062 9303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:11.914340 9303 catalog_manager.cc:1259] Loaded cluster ID: 0a48e20e0d6d4e74afedac5fb1eec1ac
I20260806 13:00:11.914381 9303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:11.915581 9303 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:11.915781 9303 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Loaded TSK: 0
I20260806 13:00:11.915838 9303 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 049b7e608ae54aefafae8104c741fec1: Loaded TSK: 1
F20260806 13:00:11.916105 9303 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f752b56c9ad google::LogMessage::Fail() at ??:0
@ 0x7f752b56f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f752b56c4de google::LogMessage::Flush() at ??:0
@ 0x7f752b56da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f752dcd9104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f752dcd6d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f75334ca1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f75334ca48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f75334ca594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f7533490c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f753349aad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f752d43a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f752d42f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f752e2936db start_thread at ??:0
@ 0x7f752a06a61f clone at ??:0
W20260806 13:00:12.005580 9295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:12.005774 9295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:12.005808 9295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:12.006954 9295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:12.006996 9295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:12.007023 9295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 13:00:12.008524 9295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:46053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=41183
--tserver_master_addrs=127.24.83.62:34107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:12.008775 9295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:12.009006 9295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:12.015100 9336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:12.015295 9295 server_base.cc:1034] running on GCE node
W20260806 13:00:12.015112 9335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:12.015100 9338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:12.015596 9295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:12.015885 9295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:12.017057 9295 hybrid_clock.cc:648] HybridClock initialized: now 1786021212017046 us; error 27 us; skew 500 ppm
Aug 06 13:00:12 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021212, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:12.019748 9295 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 13:00:12.020217 9295 webserver.cc:460] Webserver started at http://127.24.83.1:41183/ using document root <none> and password file <none>
I20260806 13:00:12.020469 9295 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:12.020536 9295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:12.021929 9295 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:12.022686 9346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:12.022862 9295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 13:00:12.022964 9295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "7fff71f6435f4c2c91ea52bc0931259f"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "371a1815ee82edfb78a5353e942181f0"
server_key_iv: "d047f18e8bd1a5aa8163b7ee98a225ff"
server_key_version: "encryptionkey@0"
I20260806 13:00:12.023327 9295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:12.030262 9295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:12.032455 9295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:12.032605 9295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:12.032825 9295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:12.033255 9354 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 13:00:12.035207 9295 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 13:00:12.035264 9295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 13:00:12.035307 9295 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 13:00:12.035997 9295 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 13:00:12.036048 9295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:12.036108 9354 tablet_bootstrap.cc:492] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: Bootstrap starting.
I20260806 13:00:12.043393 9295 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:46053
I20260806 13:00:12.043839 9295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 13:00:12.044728 9463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:46053 every 8 connection(s)
I20260806 13:00:12.047691 9354 log.cc:824] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:12.047736 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 9295
I20260806 13:00:12.048130 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:45305
--local_ip_for_outbound_sockets=127.24.83.2
--tserver_master_addrs=127.24.83.62:34107
--webserver_port=41027
--webserver_interface=127.24.83.2
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:00:12.050830 9354 tablet_bootstrap.cc:492] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:00:12.051157 9354 tablet_bootstrap.cc:492] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: Bootstrap complete.
I20260806 13:00:12.052502 9354 ts_tablet_manager.cc:1397] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.013s
I20260806 13:00:12.053978 9354 raft_consensus.cc:348] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 13:00:12.054313 9354 raft_consensus.cc:729] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fff71f6435f4c2c91ea52bc0931259f, State: Initialized, Role: FOLLOWER
I20260806 13:00:12.054482 9354 consensus_queue.cc:260] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 13:00:12.054747 9354 ts_tablet_manager.cc:1428] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260806 13:00:12.115597 9466 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:00:12.044787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:37247)
0806 13:00:12.048565 (+ 3778us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:12.048618 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:12.049399 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:12.115341 (+ 65942us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":3727,"thread_start_us":93,"threads_started":1}
W20260806 13:00:12.115963 9464 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:34107 (0 consecutive failures): Network error: Failed to ping master at 127.24.83.62:34107: Client connection negotiation failed: client connection to 127.24.83.62:34107: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 13:00:12.116338 9466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:12.116200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:53081)
0806 13:00:12.116256 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:12.116308 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 13:00:12.116788 9466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:12.116678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:41963)
0806 13:00:12.116721 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:12.116752 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 13:00:12.116905 9464 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 13:00:12.133767 9468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:12.133962 9468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:12.133986 9468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:12.135062 9468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:12.135097 9468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:12.135121 9468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 13:00:12.136567 9468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:45305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=41027
--tserver_master_addrs=127.24.83.62:34107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:12.136761 9468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:12.136943 9468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:12.141963 9475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:12.142098 9468 server_base.cc:1034] running on GCE node
W20260806 13:00:12.141965 9474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:12.141963 9477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:12.142436 9468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:12.142710 9468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:12.143925 9468 hybrid_clock.cc:648] HybridClock initialized: now 1786021212143883 us; error 36 us; skew 500 ppm
Aug 06 13:00:12 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021212, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:12.146515 9468 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 13:00:12.146955 9468 webserver.cc:460] Webserver started at http://127.24.83.2:41027/ using document root <none> and password file <none>
I20260806 13:00:12.147192 9468 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:12.147258 9468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:12.148686 9468 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:12.149359 9484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:12.149559 9468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:12.149655 9468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "9638e09612a944cab7e51f95a2016037"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "dd04611faec26138ef66d56a111c69e0"
server_key_iv: "ac03737a1c36690f02429f7b8caf5f4f"
server_key_version: "encryptionkey@0"
I20260806 13:00:12.150013 9468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:12.178823 9468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:12.180770 9468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:12.180948 9468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:12.181188 9468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:12.181511 9468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:12.181567 9468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:12.181608 9468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:12.181638 9468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:12.187522 9468 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:45305
I20260806 13:00:12.187573 9603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:45305 every 8 connection(s)
I20260806 13:00:12.187927 9468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 13:00:12.188987 9606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:12.188609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:54719)
0806 13:00:12.188769 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:12.188845 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
W20260806 13:00:12.189396 9604 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:34107 (0 consecutive failures): Network error: Failed to ping master at 127.24.83.62:34107: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
I20260806 13:00:12.189671 9606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:12.189571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:39211)
0806 13:00:12.189613 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:12.189653 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 13:00:12.189954 9606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:12.189885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:50435)
0806 13:00:12.189919 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:12.189940 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 13:00:12.190064 9604 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 13:00:12.194350 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 9468
I20260806 13:00:12.194653 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:43223
--local_ip_for_outbound_sockets=127.24.83.3
--tserver_master_addrs=127.24.83.62:34107
--webserver_port=36225
--webserver_interface=127.24.83.3
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:12.282150 9607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:12.282366 9607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:12.282390 9607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:12.283740 9607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:12.283794 9607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:12.283823 9607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 13:00:12.285385 9607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:43223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=36225
--tserver_master_addrs=127.24.83.62:34107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:12.285578 9607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:12.285766 9607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:12.291004 9613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:12.290992 9612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:12.291051 9607 server_base.cc:1034] running on GCE node
W20260806 13:00:12.290980 9615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:12.291467 9607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:12.291730 9607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:12.292910 9607 hybrid_clock.cc:648] HybridClock initialized: now 1786021212292893 us; error 29 us; skew 500 ppm
Aug 06 13:00:12 dist-test-slave-zkpd krb5kdc[8130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021212, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:12.295511 9607 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 13:00:12.296191 9607 webserver.cc:460] Webserver started at http://127.24.83.3:36225/ using document root <none> and password file <none>
I20260806 13:00:12.296442 9607 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:12.296512 9607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:12.298154 9607 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:00:12.298964 9622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:12.299229 9607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:12.299355 9607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "6e68537becb04c1c99d4981ffa922bd1"
format_stamp: "Formatted at 2026-08-06 12:59:30 on dist-test-slave-zkpd"
server_key: "ad87e76e9eef9bc4e8458cf33d498ac9"
server_key_iv: "c26e62653aafbcb1084b55e650cbccdc"
server_key_version: "encryptionkey@0"
I20260806 13:00:12.299814 9607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:12.307291 9607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:12.309494 9607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:12.309849 9607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:12.310144 9607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:12.310474 9607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:12.310534 9607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:12.310577 9607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:12.310621 9607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:12.317137 9607 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:43223
I20260806 13:00:12.317224 9740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:43223 every 8 connection(s)
I20260806 13:00:12.317611 9607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 13:00:12.318776 9743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:12.318383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:48907)
0806 13:00:12.318549 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:12.318624 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
W20260806 13:00:12.319237 9741 heartbeater.cc:644] Failed to heartbeat to 127.24.83.62:34107 (0 consecutive failures): Network error: Failed to ping master at 127.24.83.62:34107: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
I20260806 13:00:12.319525 9743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:12.319433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:48019)
0806 13:00:12.319481 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:12.319507 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 13:00:12.319831 9743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:12.319772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:47381)
0806 13:00:12.319796 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:12.319817 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 13:00:12.319962 9741 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 13:00:12.320194 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 9607
I20260806 13:00:13.117542 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:13.117277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:39853)
0806 13:00:13.117449 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:13.117490 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":65,"threads_started":1}
I20260806 13:00:13.190585 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:13.190379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:54817)
0806 13:00:13.190499 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:13.190536 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260806 13:00:13.320456 9747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:13.320257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:38307)
0806 13:00:13.320366 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:13.320404 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260806 13:00:13.812979 9748 raft_consensus.cc:482] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:00:13.813104 9748 raft_consensus.cc:504] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 13:00:13.813419 9748 leader_election.cc:288] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305), 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223)
Aug 06 13:00:13 dist-test-slave-zkpd krb5kdc[8130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021212, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.2@KRBTEST.COM
Aug 06 13:00:13 dist-test-slave-zkpd krb5kdc[8130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021212, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.3@KRBTEST.COM
I20260806 13:00:13.824086 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:13.813647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:45305 (local address 127.24.83.1:39467)
0806 13:00:13.813795 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:13.813807 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:13.813869 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:13.814747 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:13.814750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:13.815176 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:13.815391 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:13.815398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:13.816597 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:13.816598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:13.817373 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:13.817376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:13.817964 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:13.817965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:13.818095 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:13.818752 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:00:13.818761 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:13.820243 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:13.822374 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:13.822382 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:13.822384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:13.822681 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:13.822955 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:13.822956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:13.822956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:13.823083 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:13.823507 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:13.823514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:13.823654 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 13:00:13.823755 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 13:00:13.823965 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":71,"thread_start_us":73,"threads_started":1}
I20260806 13:00:13.824115 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:13.813993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:51165 (local address 127.24.83.3:43223)
0806 13:00:13.814142 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:13.814150 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:13.814176 (+ 26us) server_negotiation.cc:407] Connection header received
0806 13:00:13.814613 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:13.814614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:13.814696 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:13.814870 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:13.815522 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:13.816461 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:13.817471 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:13.817868 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:13.817990 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:13.820781 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:13.820785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:13.820785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:13.820794 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:13.822468 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:13.822883 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:13.822886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:13.822886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:13.823010 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:13.823233 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:13.823233 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:13.823234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:13.823445 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:13.823543 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:13.823865 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 13:00:13.824013 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260806 13:00:13.824191 9750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:13.813785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:39467 (local address 127.24.83.2:45305)
0806 13:00:13.813984 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:13.813996 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:13.814025 (+ 29us) server_negotiation.cc:407] Connection header received
0806 13:00:13.814496 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:13.814499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:13.814544 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:13.814668 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:13.815549 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:13.816476 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:13.817473 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:13.817869 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:13.817996 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:13.820364 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:13.820369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:13.820370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:13.820382 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:13.822287 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:13.822762 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:13.822766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:13.822766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:13.822885 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:13.823150 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:13.823151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:13.823151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:13.823433 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:13.823515 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:13.823841 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 13:00:13.824076 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260806 13:00:13.824227 9751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:13.813851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:43223 (local address 127.24.83.1:51165)
0806 13:00:13.813959 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:13.813967 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:13.814010 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:13.814845 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:13.814846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:13.815187 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:13.815392 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:13.815398 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:13.816564 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:13.816565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:13.817375 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:13.817378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:13.817967 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:13.817968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:13.818095 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:13.818816 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:00:13.818821 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:13.820669 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:13.822546 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:13.822547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:13.822547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:13.822802 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:13.823082 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:13.823083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:13.823083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:13.823163 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:13.823540 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:13.823541 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:13.823653 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 13:00:13.823803 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 13:00:13.824119 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":199,"thread_start_us":79,"threads_started":1}
W20260806 13:00:13.825518 9347 leader_election.cc:341] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223): Not found: Tablet not found: eda53d7ce8544d05a8220354f97ccf94
W20260806 13:00:13.825647 9347 leader_election.cc:341] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305): Not found: Tablet not found: eda53d7ce8544d05a8220354f97ccf94
I20260806 13:00:13.825702 9347 leader_election.cc:302] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fff71f6435f4c2c91ea52bc0931259f; no voters: 6e68537becb04c1c99d4981ffa922bd1, 9638e09612a944cab7e51f95a2016037
I20260806 13:00:13.825843 9748 raft_consensus.cc:2726] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:00:14.118302 9751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:14.118129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:39595)
0806 13:00:14.118201 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:14.118256 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 13:00:14.191349 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:14.191098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:47613)
0806 13:00:14.191236 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:14.191294 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260806 13:00:14.321231 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:14.320946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:40489)
0806 13:00:14.321111 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:14.321160 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":65,"threads_started":1}
I20260806 13:00:15.119120 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:15.118888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:57071)
0806 13:00:15.119015 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:15.119052 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260806 13:00:15.192126 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:15.191908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:34227)
0806 13:00:15.192044 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:15.192081 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260806 13:00:15.321871 9757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:15.321674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:60327)
0806 13:00:15.321800 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:15.321833 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260806 13:00:15.456692 9758 raft_consensus.cc:482] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:00:15.456809 9758 raft_consensus.cc:504] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 13:00:15.457022 9758 leader_election.cc:288] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305), 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223)
W20260806 13:00:15.457485 9347 leader_election.cc:341] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223): Not found: Tablet not found: eda53d7ce8544d05a8220354f97ccf94
W20260806 13:00:15.457573 9347 leader_election.cc:341] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305): Not found: Tablet not found: eda53d7ce8544d05a8220354f97ccf94
I20260806 13:00:15.457619 9347 leader_election.cc:302] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fff71f6435f4c2c91ea52bc0931259f; no voters: 6e68537becb04c1c99d4981ffa922bd1, 9638e09612a944cab7e51f95a2016037
I20260806 13:00:15.457719 9758 raft_consensus.cc:2726] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:00:16.120074 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:16.119802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:39109)
0806 13:00:16.119988 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:16.120023 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":63,"threads_started":1}
I20260806 13:00:16.192747 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:16.192535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:59751)
0806 13:00:16.192675 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:16.192707 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260806 13:00:16.322575 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:16.322348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:44721)
0806 13:00:16.322479 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:16.322518 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260806 13:00:17.120821 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:17.120596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:60013)
0806 13:00:17.120738 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:17.120772 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260806 13:00:17.193436 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:17.193203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:53237)
0806 13:00:17.193355 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:17.193392 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260806 13:00:17.323316 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:17.323060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:58787)
0806 13:00:17.323220 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:17.323270 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260806 13:00:17.445614 9765 raft_consensus.cc:482] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:00:17.445712 9765 raft_consensus.cc:504] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 13:00:17.445916 9765 leader_election.cc:288] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305), 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223)
W20260806 13:00:17.446311 9347 leader_election.cc:341] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305): Not found: Tablet not found: eda53d7ce8544d05a8220354f97ccf94
W20260806 13:00:17.446390 9347 leader_election.cc:341] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223): Not found: Tablet not found: eda53d7ce8544d05a8220354f97ccf94
I20260806 13:00:17.446437 9347 leader_election.cc:302] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fff71f6435f4c2c91ea52bc0931259f; no voters: 6e68537becb04c1c99d4981ffa922bd1, 9638e09612a944cab7e51f95a2016037
I20260806 13:00:17.446547 9765 raft_consensus.cc:2726] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:00:18.121513 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:18.121292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:59727)
0806 13:00:18.121429 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:18.121464 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260806 13:00:18.194146 9767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:18.193934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:35123)
0806 13:00:18.194065 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:18.194106 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260806 13:00:18.324100 9768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:18.323829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:47621)
0806 13:00:18.324019 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:18.324060 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":123,"threads_started":1}
I20260806 13:00:19.122306 9769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:19.122058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:32957)
0806 13:00:19.122206 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:19.122254 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260806 13:00:19.194814 9771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:19.194614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:37305)
0806 13:00:19.194741 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:19.194774 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260806 13:00:19.324787 9772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:19.324566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:40659)
0806 13:00:19.324704 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:19.324746 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260806 13:00:20.123189 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:20.122925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:59915)
0806 13:00:20.123099 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:20.123137 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260806 13:00:20.195493 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:20.195302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:39913)
0806 13:00:20.195418 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:20.195452 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260806 13:00:20.325488 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:20.325246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:51787)
0806 13:00:20.325395 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:20.325432 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260806 13:00:20.836449 9777 raft_consensus.cc:482] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:00:20.836541 9777 raft_consensus.cc:504] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 13:00:20.836711 9777 leader_election.cc:288] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305), 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223)
W20260806 13:00:20.837136 9347 leader_election.cc:341] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305): Not found: Tablet not found: eda53d7ce8544d05a8220354f97ccf94
W20260806 13:00:20.837217 9347 leader_election.cc:341] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223): Not found: Tablet not found: eda53d7ce8544d05a8220354f97ccf94
I20260806 13:00:20.837246 9347 leader_election.cc:302] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fff71f6435f4c2c91ea52bc0931259f; no voters: 6e68537becb04c1c99d4981ffa922bd1, 9638e09612a944cab7e51f95a2016037
I20260806 13:00:20.837318 9777 raft_consensus.cc:2726] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:00:21.123976 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:21.123686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:34873)
0806 13:00:21.123839 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:21.123876 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260806 13:00:21.196167 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:21.195976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:50087)
0806 13:00:21.196093 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:21.196125 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260806 13:00:21.326131 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:21.325921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:43823)
0806 13:00:21.326062 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:21.326092 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260806 13:00:22.124691 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:22.124475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:37603)
0806 13:00:22.124605 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:22.124640 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260806 13:00:22.196807 9782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:22.196573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:47825)
0806 13:00:22.196729 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:22.196767 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":57,"threads_started":1}
I20260806 13:00:22.326819 9784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:22.326571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:58007)
0806 13:00:22.326708 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:22.326747 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260806 13:00:23.125352 9785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:23.125125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:33387)
0806 13:00:23.125265 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:23.125301 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260806 13:00:23.197463 9786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:23.197262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:58339)
0806 13:00:23.197385 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:23.197421 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260806 13:00:23.327436 9787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:23.327240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:52105)
0806 13:00:23.327356 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:23.327391 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260806 13:00:24.126286 9788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:24.126038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:44473)
0806 13:00:24.126173 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:24.126234 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260806 13:00:24.198096 9789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:24.197892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:48531)
0806 13:00:24.198017 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:24.198054 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260806 13:00:24.328025 9790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:24.327808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:54057)
0806 13:00:24.327951 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:24.327983 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":59,"threads_started":1}
I20260806 13:00:25.126943 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:25.126713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:33851)
0806 13:00:25.126854 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:25.126895 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260806 13:00:25.198993 9792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:25.198742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:38673)
0806 13:00:25.198905 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:25.198949 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260806 13:00:25.328676 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:25.328459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:56107)
0806 13:00:25.328599 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:25.328632 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260806 13:00:26.127583 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:26.127374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:35039)
0806 13:00:26.127504 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:26.127537 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260806 13:00:26.199702 9795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:26.199489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:45687)
0806 13:00:26.199632 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:26.199658 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":103,"threads_started":1}
I20260806 13:00:26.329455 9796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:26.329208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:52445)
0806 13:00:26.329366 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:26.329406 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":61,"threads_started":1}
I20260806 13:00:26.699246 9797 raft_consensus.cc:482] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:00:26.699343 9797 raft_consensus.cc:504] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9638e09612a944cab7e51f95a2016037" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45305 } } peers { permanent_uuid: "6e68537becb04c1c99d4981ffa922bd1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43223 } } peers { permanent_uuid: "7fff71f6435f4c2c91ea52bc0931259f" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 46053 } }
I20260806 13:00:26.699502 9797 leader_election.cc:288] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305), 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223)
W20260806 13:00:26.699959 9347 leader_election.cc:341] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9638e09612a944cab7e51f95a2016037 (127.24.83.2:45305): Not found: Tablet not found: eda53d7ce8544d05a8220354f97ccf94
W20260806 13:00:26.700066 9347 leader_election.cc:341] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e68537becb04c1c99d4981ffa922bd1 (127.24.83.3:43223): Not found: Tablet not found: eda53d7ce8544d05a8220354f97ccf94
I20260806 13:00:26.700101 9347 leader_election.cc:302] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fff71f6435f4c2c91ea52bc0931259f; no voters: 6e68537becb04c1c99d4981ffa922bd1, 9638e09612a944cab7e51f95a2016037
I20260806 13:00:26.700196 9797 raft_consensus.cc:2726] T eda53d7ce8544d05a8220354f97ccf94 P 7fff71f6435f4c2c91ea52bc0931259f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:00:27.128329 9798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:27.128125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.1:39707)
0806 13:00:27.128249 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:27.128283 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260806 13:00:27.200286 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:27.200100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.2:58205)
0806 13:00:27.200212 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:27.200244 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260806 13:00:27.324286 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 9295
I20260806 13:00:27.330027 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 9468
I20260806 13:00:27.330188 9800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:27.329978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:34107 (local address 127.24.83.3:42381)
0806 13:00:27.330098 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:27.330134 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.62:34107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260806 13:00:27.335055 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 9607
2026-08-06T13:00:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58229 ms)
[----------] 30 tests from SecurityITest (235999 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:27 dist-test-slave-zkpd krb5kdc[9803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:27 dist-test-slave-zkpd krb5kdc[9803](info): set up 2 sockets
Aug 06 13:00:27 dist-test-slave-zkpd krb5kdc[9803](info): commencing operation
krb5kdc: starting...
W20260806 13:00:28.479749 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:28 dist-test-slave-zkpd krb5kdc[9803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 13:00:28.594674 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:46599
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:46599
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:28.674496 9816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:28.674683 9816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:28.674705 9816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:28.675724 9816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:28.675757 9816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:28.675784 9816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:28.675796 9816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:28.675807 9816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:28.677179 9816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:46599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:46599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:28.677363 9816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:28.677520 9816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:28.682523 9822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:28.682546 9816 server_base.cc:1034] running on GCE node
W20260806 13:00:28.682489 9821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:28.682507 9824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:28.682981 9816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:28.683257 9816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:28.684417 9816 hybrid_clock.cc:648] HybridClock initialized: now 1786021228684407 us; error 39 us; skew 500 ppm
Aug 06 13:00:28 dist-test-slave-zkpd krb5kdc[9803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021228, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:28.686777 9816 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:00:28.687191 9816 webserver.cc:460] Webserver started at http://127.24.83.62:43681/ using document root <none> and password file <none>
I20260806 13:00:28.687427 9816 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:28.687505 9816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:28.687646 9816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:28.688696 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "8d273c161cf24cc8896e7809c38fdbaf"
format_stamp: "Formatted at 2026-08-06 13:00:28 on dist-test-slave-zkpd"
server_key: "921e297209751ca41e92dc0bfbb28ad5"
server_key_iv: "a3e59913183381c3c715f60a5bb0b2b8"
server_key_version: "encryptionkey@0"
I20260806 13:00:28.689081 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "8d273c161cf24cc8896e7809c38fdbaf"
format_stamp: "Formatted at 2026-08-06 13:00:28 on dist-test-slave-zkpd"
server_key: "921e297209751ca41e92dc0bfbb28ad5"
server_key_iv: "a3e59913183381c3c715f60a5bb0b2b8"
server_key_version: "encryptionkey@0"
I20260806 13:00:28.690366 9816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:00:28.691061 9832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:28.691227 9816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:00:28.691327 9816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "8d273c161cf24cc8896e7809c38fdbaf"
format_stamp: "Formatted at 2026-08-06 13:00:28 on dist-test-slave-zkpd"
server_key: "921e297209751ca41e92dc0bfbb28ad5"
server_key_iv: "a3e59913183381c3c715f60a5bb0b2b8"
server_key_version: "encryptionkey@0"
I20260806 13:00:28.691418 9816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:28.707167 9816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:28.708988 9816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:28.709149 9816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:28.713111 9816 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:46599
I20260806 13:00:28.713147 9885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:46599 every 8 connection(s)
I20260806 13:00:28.713482 9816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:28.714334 9886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:28.716620 9886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf: Bootstrap starting.
I20260806 13:00:28.717193 9886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:28.717476 9886 log.cc:824] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:28.718164 9886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf: No bootstrap required, opened a new log
I20260806 13:00:28.719465 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 9816
I20260806 13:00:28.719584 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:28.719776 24908 external_mini_cluster.cc:1428] Setting key b8340358235f368e34b8f621d198a0ff
I20260806 13:00:28.720163 9886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d273c161cf24cc8896e7809c38fdbaf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46599 } }
I20260806 13:00:28.720366 9886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:28.720446 9886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d273c161cf24cc8896e7809c38fdbaf, State: Initialized, Role: FOLLOWER
I20260806 13:00:28.720615 9886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d273c161cf24cc8896e7809c38fdbaf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46599 } }
I20260806 13:00:28.720718 9886 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:28.720763 9886 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:28.720813 9886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:28.721939 9886 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d273c161cf24cc8896e7809c38fdbaf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46599 } }
I20260806 13:00:28.722025 9886 leader_election.cc:302] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d273c161cf24cc8896e7809c38fdbaf; no voters:
I20260806 13:00:28.722208 9886 leader_election.cc:288] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:28.722226 9891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:28.722342 9891 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [term 1 LEADER]: Becoming Leader. State: Replica: 8d273c161cf24cc8896e7809c38fdbaf, State: Running, Role: LEADER
I20260806 13:00:28.722470 9886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:28.722471 9891 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d273c161cf24cc8896e7809c38fdbaf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46599 } }
I20260806 13:00:28.722934 9893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d273c161cf24cc8896e7809c38fdbaf. Latest consensus state: current_term: 1 leader_uuid: "8d273c161cf24cc8896e7809c38fdbaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d273c161cf24cc8896e7809c38fdbaf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46599 } } }
I20260806 13:00:28.723208 9893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:28.723130 9892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d273c161cf24cc8896e7809c38fdbaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d273c161cf24cc8896e7809c38fdbaf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46599 } } }
I20260806 13:00:28.723713 9892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:28.723923 9900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:28.724329 9900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:28.726507 9900 catalog_manager.cc:1347] Generated new cluster ID: 326be0cbc1c541ae9070d9603e393ed3
I20260806 13:00:28.726564 9900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:00:28 dist-test-slave-zkpd krb5kdc[9803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:28.730551 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:28.720329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42704 (local address 127.24.83.62:46599)
0806 13:00:28.720540 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:28.720548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:28.720572 (+ 24us) server_negotiation.cc:407] Connection header received
0806 13:00:28.720662 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:28.720667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:28.720716 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:28.720868 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:28.721628 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:28.722749 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:28.723862 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:28.724481 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:28.724592 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:28.727311 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:28.727315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:28.727315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:28.727326 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:28.729076 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:28.729448 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:28.729451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:28.729451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:28.729568 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:28.729793 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:28.729794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:28.729794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:28.730016 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:28.730191 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:28.730268 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 13:00:28.730403 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":107,"threads_started":1}
I20260806 13:00:28.730851 9900 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:28.731500 9900 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:28.737295 9900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf: Generated new TSK 0
I20260806 13:00:28.737461 9900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 13:00:28.770788 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46599
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:28.851714 9912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:28.851991 9912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:28.852032 9912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:28.853190 9912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:28.853252 9912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:28.853299 9912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 13:00:28.854738 9912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:28.854979 9912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:28.855159 9912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:28.859995 9918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:28.860049 9912 server_base.cc:1034] running on GCE node
W20260806 13:00:28.860035 9919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:28.860023 9921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:28.860343 9912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:28.860618 9912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:28.861788 9912 hybrid_clock.cc:648] HybridClock initialized: now 1786021228861790 us; error 38 us; skew 500 ppm
Aug 06 13:00:28 dist-test-slave-zkpd krb5kdc[9803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021228, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:28.864156 9912 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 13:00:28.864605 9912 webserver.cc:460] Webserver started at http://127.24.83.1:45017/ using document root <none> and password file <none>
I20260806 13:00:28.864854 9912 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:28.864921 9912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:28.865075 9912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:28.866096 9912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "89ff62549ec340d2a62730101cc8e43e"
format_stamp: "Formatted at 2026-08-06 13:00:28 on dist-test-slave-zkpd"
server_key: "49652d68d473befb39a8a5d23bd74dea"
server_key_iv: "82fd025632fbaab9a6472a8d5563d442"
server_key_version: "encryptionkey@0"
I20260806 13:00:28.866485 9912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "89ff62549ec340d2a62730101cc8e43e"
format_stamp: "Formatted at 2026-08-06 13:00:28 on dist-test-slave-zkpd"
server_key: "49652d68d473befb39a8a5d23bd74dea"
server_key_iv: "82fd025632fbaab9a6472a8d5563d442"
server_key_version: "encryptionkey@0"
I20260806 13:00:28.867983 9912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:00:28.868741 9928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:28.868934 9912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:28.869035 9912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "89ff62549ec340d2a62730101cc8e43e"
format_stamp: "Formatted at 2026-08-06 13:00:28 on dist-test-slave-zkpd"
server_key: "49652d68d473befb39a8a5d23bd74dea"
server_key_iv: "82fd025632fbaab9a6472a8d5563d442"
server_key_version: "encryptionkey@0"
I20260806 13:00:28.869122 9912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:28.883620 9912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:28.885380 9912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:28.885528 9912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:28.885738 9912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:28.886088 9912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:28.886149 9912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:28.886190 9912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:28.886217 9912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:28.892124 9912 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:40519
I20260806 13:00:28.892184 10044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:40519 every 8 connection(s)
I20260806 13:00:28.892500 9912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 13:00:28.895428 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 9912
I20260806 13:00:28.895546 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 13:00:28.895753 24908 external_mini_cluster.cc:1428] Setting key 634f0742fe5994d113828ff811fd67c0
Aug 06 13:00:28 dist-test-slave-zkpd krb5kdc[9803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021228, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:28.902410 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:28.893216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:59429 (local address 127.24.83.62:46599)
0806 13:00:28.893277 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:28.893279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:28.893456 (+ 177us) server_negotiation.cc:407] Connection header received
0806 13:00:28.893951 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:28.893952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:28.893964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:28.894018 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:28.894977 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:28.895662 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:28.896921 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:28.897163 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:28.897237 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:28.899513 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:28.899515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:28.899515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:28.899523 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:28.901058 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:28.901487 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:28.901488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:28.901488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:28.901547 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:28.901800 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:28.901800 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:28.901801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:28.901910 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:28.901977 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:28.902243 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 13:00:28.902321 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 13:00:28.902635 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:28.893182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:46599 (local address 127.24.83.1:59429)
0806 13:00:28.893336 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:28.893376 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:28.893867 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:28.894065 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:28.894068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:28.894705 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:28.894882 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:28.894888 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:28.895777 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:28.895778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:28.896835 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:28.896838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:28.897268 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:28.897269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:28.897406 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:28.897906 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:00:28.897914 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:28.899405 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:28.901163 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:28.901170 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:28.901172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:28.901411 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:28.901614 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:28.901615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:28.901615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:28.901723 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:28.901981 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:28.901988 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:28.902117 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 13:00:28.902219 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 13:00:28.902376 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
I20260806 13:00:28.903097 10045 heartbeater.cc:347] Connected to a master server at 127.24.83.62:46599
I20260806 13:00:28.903185 10045 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:28.903388 10045 heartbeater.cc:511] Master 127.24.83.62:46599 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
I20260806 13:00:28.904059 9850 ts_manager.cc:194] Registered new tserver with Master: 89ff62549ec340d2a62730101cc8e43e (127.24.83.1:40519)
I20260806 13:00:28.904656 9850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:59429
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 13:00:28.920924 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46599
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:29.000947 10050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:29.001135 10050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:29.001157 10050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:29.002194 10050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:29.002226 10050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:29.002251 10050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 13:00:29.003616 10050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:29.003784 10050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:29.003971 10050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 13:00:29.008775 10050 server_base.cc:1034] running on GCE node
W20260806 13:00:29.008752 10056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:29.008810 10055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:29.008759 10058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:29.009177 10050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:29.009445 10050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:29.010592 10050 hybrid_clock.cc:648] HybridClock initialized: now 1786021229010582 us; error 26 us; skew 500 ppm
Aug 06 13:00:29 dist-test-slave-zkpd krb5kdc[9803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021229, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:29.013105 10050 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 13:00:29.013473 10050 webserver.cc:460] Webserver started at http://127.24.83.2:42633/ using document root <none> and password file <none>
I20260806 13:00:29.013643 10050 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:29.013696 10050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:29.013783 10050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:29.014711 10050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "9ea22bc658954492910b0f9bb8321450"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "2da2293947214f0047913173206ef667"
server_key_iv: "179922aaffee08f73d5398f451d50b8f"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.015089 10050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "9ea22bc658954492910b0f9bb8321450"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "2da2293947214f0047913173206ef667"
server_key_iv: "179922aaffee08f73d5398f451d50b8f"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.016412 10050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:00:29.017068 10065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.017236 10050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:00:29.017347 10050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "9ea22bc658954492910b0f9bb8321450"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "2da2293947214f0047913173206ef667"
server_key_iv: "179922aaffee08f73d5398f451d50b8f"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.017421 10050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:29.032501 10050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:29.034833 10050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:29.034992 10050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:29.035254 10050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:29.035573 10050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:29.035626 10050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.035669 10050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:29.035696 10050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.041385 10050 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:37579
I20260806 13:00:29.041471 10180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:37579 every 8 connection(s)
I20260806 13:00:29.041764 10050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 13:00:29.045470 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 10050
I20260806 13:00:29.045584 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 13:00:29.045804 24908 external_mini_cluster.cc:1428] Setting key 078803136d0b652a6dbb1b590a44dc4d
Aug 06 13:00:29 dist-test-slave-zkpd krb5kdc[9803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021229, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:29.051401 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.042430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:58305 (local address 127.24.83.62:46599)
0806 13:00:29.042474 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.042476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.042666 (+ 190us) server_negotiation.cc:407] Connection header received
0806 13:00:29.043222 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.043224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.043235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.043286 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:29.044397 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.044881 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.045956 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.046182 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.046240 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.048565 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:29.048566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:29.048566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:29.048572 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:29.049996 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:29.050487 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:29.050488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:29.050488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:29.050535 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:29.050794 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:29.050794 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:29.050794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:29.050887 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:29.050938 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.051266 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.051332 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:00:29.051697 10184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.042405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:46599 (local address 127.24.83.2:58305)
0806 13:00:29.042544 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.042584 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.043075 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.043356 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.043360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.044121 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:29.044310 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.044315 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.044965 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.044965 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.045848 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.045850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.046253 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.046253 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.046396 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.046915 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:00:29.046923 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:29.048450 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:29.050079 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:29.050086 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:29.050088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:29.050412 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:29.050598 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:29.050598 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:29.050599 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:29.050731 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:29.050958 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:29.050966 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:29.051126 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.051250 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.051430 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260806 13:00:29.052233 10182 heartbeater.cc:347] Connected to a master server at 127.24.83.62:46599
I20260806 13:00:29.052340 10182 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:29.052523 10182 heartbeater.cc:511] Master 127.24.83.62:46599 requested a full tablet report, sending...
I20260806 13:00:29.052861 9850 ts_manager.cc:194] Registered new tserver with Master: 9ea22bc658954492910b0f9bb8321450 (127.24.83.2:37579)
I20260806 13:00:29.053222 9850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:58305
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 13:00:29.071316 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46599
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:29.164250 10187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:29.164453 10187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:29.164475 10187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:29.165627 10187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:29.165668 10187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:29.165727 10187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 13:00:29.167156 10187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:46599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:29.167413 10187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:29.167654 10187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:29.172672 10192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:29.172683 10193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:29.172724 10187 server_base.cc:1034] running on GCE node
W20260806 13:00:29.172817 10195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:29.173060 10187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:29.173322 10187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:29.174485 10187 hybrid_clock.cc:648] HybridClock initialized: now 1786021229174473 us; error 49 us; skew 500 ppm
Aug 06 13:00:29 dist-test-slave-zkpd krb5kdc[9803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021229, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:29.176930 10187 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 13:00:29.177380 10187 webserver.cc:460] Webserver started at http://127.24.83.3:43317/ using document root <none> and password file <none>
I20260806 13:00:29.177610 10187 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:29.177676 10187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:29.177829 10187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:29.178817 10187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "63f0c4bd73464996b9952c9aeaaeffe1"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "c8121b0b21d8a06b47d9df233edd74de"
server_key_iv: "7c15c0de0f4b1546196b323985cdcb05"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.179171 10187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "63f0c4bd73464996b9952c9aeaaeffe1"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "c8121b0b21d8a06b47d9df233edd74de"
server_key_iv: "7c15c0de0f4b1546196b323985cdcb05"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.180554 10187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:00:29.181546 10202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.181779 10187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:29.181893 10187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "63f0c4bd73464996b9952c9aeaaeffe1"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "c8121b0b21d8a06b47d9df233edd74de"
server_key_iv: "7c15c0de0f4b1546196b323985cdcb05"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.181968 10187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:29.200305 10187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:29.202313 10187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:29.202451 10187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:29.202677 10187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:29.202994 10187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:29.203049 10187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.203088 10187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:29.203116 10187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.209278 10187 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:43833
I20260806 13:00:29.209331 10319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:43833 every 8 connection(s)
I20260806 13:00:29.209617 10187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
Aug 06 13:00:29 dist-test-slave-zkpd krb5kdc[9803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021229, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:29.216245 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 10187
I20260806 13:00:29.216373 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 13:00:29.216585 24908 external_mini_cluster.cc:1428] Setting key e23831210bf28a416df3f50914f75ef4
I20260806 13:00:29.218546 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.210311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:55147 (local address 127.24.83.62:46599)
0806 13:00:29.210390 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.210393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.210573 (+ 180us) server_negotiation.cc:407] Connection header received
0806 13:00:29.211011 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.211012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.211022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.211074 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:29.212003 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.212496 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.213439 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.213619 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.213671 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.215637 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:29.215639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:29.215639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:29.215647 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:29.217324 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:29.217779 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:29.217780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:29.217780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:29.217833 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:29.218041 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:29.218042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:29.218043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:29.218166 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:29.218224 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.218422 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.218483 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 13:00:29.218695 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.210270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:46599 (local address 127.24.83.3:55147)
0806 13:00:29.210449 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.210483 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.210929 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.211124 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.211127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.211676 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:29.211860 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.211865 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.212568 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.212569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.213358 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.213360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.213700 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.213701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.213797 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.214219 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:00:29.214227 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:29.215541 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:29.217410 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:29.217416 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:29.217418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:29.217712 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:29.217877 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:29.217877 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:29.217878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:29.217985 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:29.218215 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:29.218217 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:29.218319 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.218394 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.218502 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
I20260806 13:00:29.219131 10320 heartbeater.cc:347] Connected to a master server at 127.24.83.62:46599
I20260806 13:00:29.219231 10320 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:29.219403 10320 heartbeater.cc:511] Master 127.24.83.62:46599 requested a full tablet report, sending...
I20260806 13:00:29.219770 9850 ts_manager.cc:194] Registered new tserver with Master: 63f0c4bd73464996b9952c9aeaaeffe1 (127.24.83.3:43833)
I20260806 13:00:29.220180 9850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:55147
I20260806 13:00:29.228457 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:00:29.235399 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.229745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.24.83.62:46599)
0806 13:00:29.229812 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.229814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.229861 (+ 47us) server_negotiation.cc:407] Connection header received
0806 13:00:29.229913 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.229914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.229922 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.229965 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:29.230518 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.230988 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.231731 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.231996 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.232077 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.233146 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:29.233147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:29.233147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:29.233155 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:29.234237 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:29.234587 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:29.234588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:29.234588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:29.234649 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:29.234860 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:29.234861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:29.234861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:29.235007 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:29.235113 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.235219 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.235299 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 13:00:29.236832 9850 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:00:29.238523 9850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:00:29.244436 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.240645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:43833 (local address 127.0.0.1:36472)
0806 13:00:29.240945 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.240954 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.240996 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.241274 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.241277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.241279 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:29.241341 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.241343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.242324 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.242325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.243235 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.243237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.244142 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.244143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.244223 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.244237 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.244311 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.244363 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260806 13:00:29.244455 10335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.240954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36472 (local address 127.24.83.3:43833)
0806 13:00:29.241122 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.241126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.241133 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:00:29.241162 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.241163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.241174 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.241259 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:29.241462 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.242232 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.243355 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.244060 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.244136 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.244189 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.244334 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.244374 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260806 13:00:29.245110 10332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.240645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:40519 (local address 127.0.0.1:54782)
0806 13:00:29.240783 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.240825 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.240902 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.241576 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.241577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.241579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:29.241632 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.241634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.242971 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.242972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.243929 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.243932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.244848 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.244849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.244928 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.244933 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.244988 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.245033 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":50,"threads_started":1}
I20260806 13:00:29.245338 10333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.240771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.24.83.1:40519)
0806 13:00:29.241312 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.241317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.241328 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:00:29.241371 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.241373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.241389 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.241716 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:29.241801 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.242864 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.244003 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.244743 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.244842 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.244896 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.245221 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.245262 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":80,"threads_started":1}
I20260806 13:00:29.245536 10252 tablet_service.cc:1467] Processing CreateTablet for tablet f435d1ddc0da490386a97a7f1b12b049 (DEFAULT_TABLE table=test-table [id=9f340583e9ec484fbffad2e15e0bf44a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:29.245929 10252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f435d1ddc0da490386a97a7f1b12b049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:29.246043 9977 tablet_service.cc:1467] Processing CreateTablet for tablet f435d1ddc0da490386a97a7f1b12b049 (DEFAULT_TABLE table=test-table [id=9f340583e9ec484fbffad2e15e0bf44a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:29.246255 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.240832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:37579 (local address 127.0.0.1:57054)
0806 13:00:29.241116 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.241126 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.241179 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.241625 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.241626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.241629 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:29.241666 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.241668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.243965 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.243966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.245060 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.245063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.246021 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.246022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.246086 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.246090 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.246151 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.246189 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":60,"threads_started":1}
I20260806 13:00:29.246384 9977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f435d1ddc0da490386a97a7f1b12b049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:29.246434 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.240955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57054 (local address 127.24.83.2:37579)
0806 13:00:29.241396 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.241401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.241411 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:00:29.241450 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.241453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.241467 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.241542 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:29.241749 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.242860 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.245148 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.245860 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.245932 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.245984 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.246186 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.246233 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":87,"threads_started":1}
I20260806 13:00:29.247112 10114 tablet_service.cc:1467] Processing CreateTablet for tablet f435d1ddc0da490386a97a7f1b12b049 (DEFAULT_TABLE table=test-table [id=9f340583e9ec484fbffad2e15e0bf44a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:29.247437 10114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f435d1ddc0da490386a97a7f1b12b049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:29.248926 10337 tablet_bootstrap.cc:492] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e: Bootstrap starting.
I20260806 13:00:29.249261 10338 tablet_bootstrap.cc:492] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1: Bootstrap starting.
I20260806 13:00:29.249617 10337 tablet_bootstrap.cc:654] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:29.249936 10338 tablet_bootstrap.cc:654] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:29.250247 10337 log.cc:824] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:29.250528 10338 log.cc:824] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:29.250998 10340 tablet_bootstrap.cc:492] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: Bootstrap starting.
I20260806 13:00:29.251226 10337 tablet_bootstrap.cc:492] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e: No bootstrap required, opened a new log
I20260806 13:00:29.251288 10337 ts_tablet_manager.cc:1397] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:00:29.251581 10340 tablet_bootstrap.cc:654] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:29.252117 10340 log.cc:824] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:29.252537 10338 tablet_bootstrap.cc:492] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1: No bootstrap required, opened a new log
I20260806 13:00:29.252621 10338 ts_tablet_manager.cc:1397] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 13:00:29.252918 10340 tablet_bootstrap.cc:492] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: No bootstrap required, opened a new log
I20260806 13:00:29.252986 10340 ts_tablet_manager.cc:1397] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:29.253232 10337 raft_consensus.cc:348] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ff62549ec340d2a62730101cc8e43e" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40519 } } peers { permanent_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43833 } } peers { permanent_uuid: "9ea22bc658954492910b0f9bb8321450" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37579 } }
I20260806 13:00:29.253396 10337 raft_consensus.cc:374] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:29.253456 10337 raft_consensus.cc:729] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ff62549ec340d2a62730101cc8e43e, State: Initialized, Role: FOLLOWER
I20260806 13:00:29.253582 10337 consensus_queue.cc:260] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ff62549ec340d2a62730101cc8e43e" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40519 } } peers { permanent_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43833 } } peers { permanent_uuid: "9ea22bc658954492910b0f9bb8321450" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37579 } }
I20260806 13:00:29.253876 10337 ts_tablet_manager.cc:1428] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:00:29.254006 10045 heartbeater.cc:503] Master 127.24.83.62:46599 was elected leader, sending a full tablet report...
I20260806 13:00:29.254606 10338 raft_consensus.cc:348] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ff62549ec340d2a62730101cc8e43e" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40519 } } peers { permanent_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43833 } } peers { permanent_uuid: "9ea22bc658954492910b0f9bb8321450" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37579 } }
I20260806 13:00:29.254776 10338 raft_consensus.cc:374] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:29.254820 10338 raft_consensus.cc:729] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f0c4bd73464996b9952c9aeaaeffe1, State: Initialized, Role: FOLLOWER
I20260806 13:00:29.254936 10338 consensus_queue.cc:260] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ff62549ec340d2a62730101cc8e43e" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40519 } } peers { permanent_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43833 } } peers { permanent_uuid: "9ea22bc658954492910b0f9bb8321450" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37579 } }
I20260806 13:00:29.254938 10340 raft_consensus.cc:348] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ff62549ec340d2a62730101cc8e43e" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40519 } } peers { permanent_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43833 } } peers { permanent_uuid: "9ea22bc658954492910b0f9bb8321450" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37579 } }
I20260806 13:00:29.255103 10340 raft_consensus.cc:374] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:29.255139 10340 raft_consensus.cc:729] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea22bc658954492910b0f9bb8321450, State: Initialized, Role: FOLLOWER
I20260806 13:00:29.255208 10338 ts_tablet_manager.cc:1428] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:00:29.255245 10320 heartbeater.cc:503] Master 127.24.83.62:46599 was elected leader, sending a full tablet report...
I20260806 13:00:29.255267 10340 consensus_queue.cc:260] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ff62549ec340d2a62730101cc8e43e" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40519 } } peers { permanent_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43833 } } peers { permanent_uuid: "9ea22bc658954492910b0f9bb8321450" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37579 } }
I20260806 13:00:29.255571 10182 heartbeater.cc:503] Master 127.24.83.62:46599 was elected leader, sending a full tablet report...
I20260806 13:00:29.255586 10340 ts_tablet_manager.cc:1428] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 13:00:29.287024 10343 raft_consensus.cc:482] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:00:29.287120 10343 raft_consensus.cc:504] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ff62549ec340d2a62730101cc8e43e" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40519 } } peers { permanent_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43833 } } peers { permanent_uuid: "9ea22bc658954492910b0f9bb8321450" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37579 } }
I20260806 13:00:29.287632 10343 leader_election.cc:288] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63f0c4bd73464996b9952c9aeaaeffe1 (127.24.83.3:43833), 9ea22bc658954492910b0f9bb8321450 (127.24.83.2:37579)
I20260806 13:00:29.291354 10335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.287976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:56315 (local address 127.24.83.3:43833)
0806 13:00:29.288017 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.288019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.288091 (+ 72us) server_negotiation.cc:407] Connection header received
0806 13:00:29.288192 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.288193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.288206 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.288240 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:29.288408 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.289136 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.290343 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.290937 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.291008 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.291019 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.291240 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.291283 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:00:29.291353 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.287864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:58977 (local address 127.24.83.2:37579)
0806 13:00:29.287928 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.287930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.287941 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:00:29.287969 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.287970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.287982 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.288033 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:29.288206 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.288918 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.290290 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.290932 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.291008 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.291020 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.291244 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.291290 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:00:29.291375 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.287770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:37579 (local address 127.24.83.1:58977)
0806 13:00:29.287827 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.287839 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.287880 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.288044 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.288045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.288048 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:29.288127 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.288129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.289010 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.289010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.290193 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.290196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.291055 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.291057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.291152 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.291154 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.291232 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.291273 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":2}
I20260806 13:00:29.291388 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.287780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:43833 (local address 127.24.83.1:56315)
0806 13:00:29.288037 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.288046 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.288114 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.288291 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.288292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.288293 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:29.288341 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.288342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.289203 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.289204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.290268 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.290270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.291055 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.291057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.291148 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.291152 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.291224 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.291273 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":16,"thread_start_us":128,"threads_started":1}
I20260806 13:00:29.291779 10135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f435d1ddc0da490386a97a7f1b12b049" candidate_uuid: "89ff62549ec340d2a62730101cc8e43e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea22bc658954492910b0f9bb8321450" is_pre_election: true
I20260806 13:00:29.291779 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f435d1ddc0da490386a97a7f1b12b049" candidate_uuid: "89ff62549ec340d2a62730101cc8e43e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" is_pre_election: true
I20260806 13:00:29.291951 10273 raft_consensus.cc:2454] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89ff62549ec340d2a62730101cc8e43e in term 0.
I20260806 13:00:29.291951 10135 raft_consensus.cc:2454] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89ff62549ec340d2a62730101cc8e43e in term 0.
I20260806 13:00:29.292199 9932 leader_election.cc:302] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63f0c4bd73464996b9952c9aeaaeffe1, 89ff62549ec340d2a62730101cc8e43e; no voters:
W20260806 13:00:29.292371 10183 tablet.cc:2369] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:00:29.292377 10343 raft_consensus.cc:2781] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:00:29.292448 10343 raft_consensus.cc:482] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:00:29.292470 10343 raft_consensus.cc:3037] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:29.293182 10343 raft_consensus.cc:504] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ff62549ec340d2a62730101cc8e43e" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40519 } } peers { permanent_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43833 } } peers { permanent_uuid: "9ea22bc658954492910b0f9bb8321450" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37579 } }
I20260806 13:00:29.293326 10343 leader_election.cc:288] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [CANDIDATE]: Term 1 election: Requested vote from peers 63f0c4bd73464996b9952c9aeaaeffe1 (127.24.83.3:43833), 9ea22bc658954492910b0f9bb8321450 (127.24.83.2:37579)
I20260806 13:00:29.293455 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f435d1ddc0da490386a97a7f1b12b049" candidate_uuid: "89ff62549ec340d2a62730101cc8e43e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63f0c4bd73464996b9952c9aeaaeffe1"
I20260806 13:00:29.293514 10135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f435d1ddc0da490386a97a7f1b12b049" candidate_uuid: "89ff62549ec340d2a62730101cc8e43e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea22bc658954492910b0f9bb8321450"
I20260806 13:00:29.293537 10273 raft_consensus.cc:3037] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:29.293620 10135 raft_consensus.cc:3037] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:29.294319 10273 raft_consensus.cc:2454] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ff62549ec340d2a62730101cc8e43e in term 1.
I20260806 13:00:29.294471 9932 leader_election.cc:302] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63f0c4bd73464996b9952c9aeaaeffe1, 89ff62549ec340d2a62730101cc8e43e; no voters:
I20260806 13:00:29.294474 10135 raft_consensus.cc:2454] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ff62549ec340d2a62730101cc8e43e in term 1.
I20260806 13:00:29.294572 10343 raft_consensus.cc:2781] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:29.294752 10343 raft_consensus.cc:686] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [term 1 LEADER]: Becoming Leader. State: Replica: 89ff62549ec340d2a62730101cc8e43e, State: Running, Role: LEADER
I20260806 13:00:29.294857 10343 consensus_queue.cc:237] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ff62549ec340d2a62730101cc8e43e" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40519 } } peers { permanent_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43833 } } peers { permanent_uuid: "9ea22bc658954492910b0f9bb8321450" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37579 } }
I20260806 13:00:29.295697 9849 catalog_manager.cc:5495] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e reported cstate change: term changed from 0 to 1, leader changed from <none> to 89ff62549ec340d2a62730101cc8e43e (127.24.83.1). New cstate: current_term: 1 leader_uuid: "89ff62549ec340d2a62730101cc8e43e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ff62549ec340d2a62730101cc8e43e" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 40519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ea22bc658954492910b0f9bb8321450" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37579 } health_report { overall_health: UNKNOWN } } }
I20260806 13:00:29.304363 10333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.301951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54790 (local address 127.24.83.1:40519)
0806 13:00:29.301993 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.301995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.302002 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:00:29.302057 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.302058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.302076 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.302169 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:00:29.302294 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.302752 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.303533 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.303744 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.303828 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.304003 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:00:29.304110 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:00:29.304198 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.304242 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.304264 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:00:29.306041 10135 raft_consensus.cc:1261] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450 [term 1 FOLLOWER]: Refusing update from remote peer 89ff62549ec340d2a62730101cc8e43e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:29.306071 10273 raft_consensus.cc:1261] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1 [term 1 FOLLOWER]: Refusing update from remote peer 89ff62549ec340d2a62730101cc8e43e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:29.306434 10347 consensus_queue.cc:1035] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ea22bc658954492910b0f9bb8321450" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:29.306586 10343 consensus_queue.cc:1035] T f435d1ddc0da490386a97a7f1b12b049 P 89ff62549ec340d2a62730101cc8e43e [LEADER]: Connected to new peer: Peer: permanent_uuid: "63f0c4bd73464996b9952c9aeaaeffe1" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 43833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:29.308993 10351 mvcc.cc:205] Tried to move back new op lower bound from 7315542955234942976 to 7315542955192406016. Current Snapshot: MvccSnapshot[applied={T|T < 7315542955234942976}]
I20260806 13:00:29.311741 9849 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:29.311857 9849 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:29.312666 9849 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d273c161cf24cc8896e7809c38fdbaf: Sending DeleteTablet for 3 replicas of tablet f435d1ddc0da490386a97a7f1b12b049
I20260806 13:00:29.313053 10114 tablet_service.cc:1514] Processing DeleteTablet for tablet f435d1ddc0da490386a97a7f1b12b049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:29 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:57054
I20260806 13:00:29.313078 10252 tablet_service.cc:1514] Processing DeleteTablet for tablet f435d1ddc0da490386a97a7f1b12b049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:29 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:36472
I20260806 13:00:29.313269 10361 tablet_replica.cc:330] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: stopping tablet replica
I20260806 13:00:29.313282 10362 tablet_replica.cc:330] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1: stopping tablet replica
I20260806 13:00:29.313277 9977 tablet_service.cc:1514] Processing DeleteTablet for tablet f435d1ddc0da490386a97a7f1b12b049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:29 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:54782
I20260806 13:00:29.313366 10362 raft_consensus.cc:2229] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:00:29.313367 10361 raft_consensus.cc:2229] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:00:29.313524 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 9912
I20260806 13:00:29.313469 10362 raft_consensus.cc:2258] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:29.314468 10362 ts_tablet_manager.cc:1905] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:00:29.315447 10361 raft_consensus.cc:2258] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:29.316386 10362 ts_tablet_manager.cc:1918] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:00:29.316465 10362 log.cc:1196] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/f435d1ddc0da490386a97a7f1b12b049
I20260806 13:00:29.316718 10362 ts_tablet_manager.cc:1939] T f435d1ddc0da490386a97a7f1b12b049 P 63f0c4bd73464996b9952c9aeaaeffe1: Deleting consensus metadata
I20260806 13:00:29.317144 9836 catalog_manager.cc:4841] TS 63f0c4bd73464996b9952c9aeaaeffe1 (127.24.83.3:43833): tablet f435d1ddc0da490386a97a7f1b12b049 (table test-table [id=9f340583e9ec484fbffad2e15e0bf44a]) successfully deleted
I20260806 13:00:29.320176 10361 ts_tablet_manager.cc:1905] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:00:29.321507 10361 ts_tablet_manager.cc:1918] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260806 13:00:29.321498 9833 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:40519 (error 108)
I20260806 13:00:29.321599 10361 log.cc:1196] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/f435d1ddc0da490386a97a7f1b12b049
I20260806 13:00:29.321749 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 10050
I20260806 13:00:29.321910 10361 ts_tablet_manager.cc:1939] T f435d1ddc0da490386a97a7f1b12b049 P 9ea22bc658954492910b0f9bb8321450: Deleting consensus metadata
I20260806 13:00:29.322474 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.321929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:40519 (local address 127.0.0.1:54800)
0806 13:00:29.322389 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.322433 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436}
W20260806 13:00:29.322809 9833 catalog_manager.cc:4584] TS 89ff62549ec340d2a62730101cc8e43e (127.24.83.1:40519): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f435d1ddc0da490386a97a7f1b12b049: Network error: Client connection negotiation failed: client connection to 127.24.83.1:40519: connect: Connection refused (error 111)
I20260806 13:00:29.327250 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.327132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:37579 (local address 127.0.0.1:57070)
0806 13:00:29.327183 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.327213 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.2:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 13:00:29.327322 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 10187
I20260806 13:00:29.332485 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 9816
2026-08-06T13:00:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1996 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T13:00:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:29Z Disabled control of system clock
I20260806 13:00:29.359720 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40971
--webserver_interface=127.24.83.62
--webserver_port=0
--builtin_ntp_servers=127.24.83.20:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:40971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:00:29.439373 10368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:29.439574 10368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:29.439594 10368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:29.440676 10368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:29.440711 10368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:29.440723 10368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:29.440735 10368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:29.440747 10368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:29.442005 10368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:40971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:40971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:29.442214 10368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:29.442433 10368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:29.447281 10374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:29.447288 10373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:29.447283 10376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:29.447535 10368 server_base.cc:1034] running on GCE node
I20260806 13:00:29.447880 10368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:29.448174 10368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:29.449312 10368 hybrid_clock.cc:648] HybridClock initialized: now 1786021229449303 us; error 27 us; skew 500 ppm
I20260806 13:00:29.450452 10368 webserver.cc:460] Webserver started at http://127.24.83.62:39423/ using document root <none> and password file <none>
I20260806 13:00:29.450654 10368 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:29.450695 10368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:29.450794 10368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:29.451768 10368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "61ceea18ebdf4679b0238c6de23b1305"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "2acc1994d6632cb1b0d881917ae68a5e"
server_key_iv: "c061e5e162012b89ab53661282bcfaea"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.452214 10368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "61ceea18ebdf4679b0238c6de23b1305"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "2acc1994d6632cb1b0d881917ae68a5e"
server_key_iv: "c061e5e162012b89ab53661282bcfaea"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.453508 10368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:29.454284 10382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.454464 10368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:29.454560 10368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "61ceea18ebdf4679b0238c6de23b1305"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "2acc1994d6632cb1b0d881917ae68a5e"
server_key_iv: "c061e5e162012b89ab53661282bcfaea"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.454644 10368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:29.469969 10368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:29.470270 10368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:29.470419 10368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:29.474395 10368 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:40971
I20260806 13:00:29.474440 10435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:40971 every 8 connection(s)
I20260806 13:00:29.474792 10368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:29.475770 10436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:29.478107 10436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305: Bootstrap starting.
I20260806 13:00:29.478681 10436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:29.478976 10436 log.cc:824] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:29.479627 10436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305: No bootstrap required, opened a new log
I20260806 13:00:29.481264 10436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ceea18ebdf4679b0238c6de23b1305" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40971 } }
I20260806 13:00:29.481400 10436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:29.481456 10436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61ceea18ebdf4679b0238c6de23b1305, State: Initialized, Role: FOLLOWER
I20260806 13:00:29.481599 10436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ceea18ebdf4679b0238c6de23b1305" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40971 } }
I20260806 13:00:29.481659 10436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:29.481679 10436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:29.481748 10436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:29.482272 10436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ceea18ebdf4679b0238c6de23b1305" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40971 } }
I20260806 13:00:29.482385 10436 leader_election.cc:302] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61ceea18ebdf4679b0238c6de23b1305; no voters:
I20260806 13:00:29.482573 10436 leader_election.cc:288] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:29.482636 10439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:29.482831 10436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:29.482832 10439 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [term 1 LEADER]: Becoming Leader. State: Replica: 61ceea18ebdf4679b0238c6de23b1305, State: Running, Role: LEADER
I20260806 13:00:29.483014 10439 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ceea18ebdf4679b0238c6de23b1305" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40971 } }
I20260806 13:00:29.483373 10441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61ceea18ebdf4679b0238c6de23b1305. Latest consensus state: current_term: 1 leader_uuid: "61ceea18ebdf4679b0238c6de23b1305" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ceea18ebdf4679b0238c6de23b1305" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40971 } } }
I20260806 13:00:29.483459 10440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61ceea18ebdf4679b0238c6de23b1305" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ceea18ebdf4679b0238c6de23b1305" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 40971 } } }
I20260806 13:00:29.483563 10440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:29.483784 10441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:29.483866 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 10368
I20260806 13:00:29.483994 10448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:29.484014 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:29.484223 24908 external_mini_cluster.cc:1428] Setting key 00e633befc49069b9af2abbb50cca074
I20260806 13:00:29.484495 10448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:29.486119 10448 catalog_manager.cc:1347] Generated new cluster ID: e2e7df1514b04ac1b3ae517122c6e262
I20260806 13:00:29.486171 10448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:29.493431 10457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.484680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34426 (local address 127.24.83.62:40971)
0806 13:00:29.488036 (+ 3356us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.488044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.488075 (+ 31us) server_negotiation.cc:407] Connection header received
0806 13:00:29.488180 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.488187 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.488249 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.488411 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:29.488661 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.490052 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.491240 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.491843 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.491970 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.492727 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:29.492730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:29.492731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:29.492924 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:29.492965 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:29.492979 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:29.493037 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.493150 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.493223 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3271,"thread_start_us":92,"threads_started":1}
I20260806 13:00:29.495496 10448 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:29.496182 10448 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:29.502344 10448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305: Generated new TSK 0
I20260806 13:00:29.502506 10448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:29.506815 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40971
--builtin_ntp_servers=127.24.83.20:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:00:29.587435 10460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:29.587662 10460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:29.587702 10460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:29.588833 10460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:29.588905 10460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:29.588953 10460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 13:00:29.590284 10460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:29.590535 10460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:29.590817 10460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:29.596496 10466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:29.596527 10465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:29.596666 10460 server_base.cc:1034] running on GCE node
W20260806 13:00:29.596498 10468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:29.596976 10460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:29.597226 10460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:29.598398 10460 hybrid_clock.cc:648] HybridClock initialized: now 1786021229598386 us; error 28 us; skew 500 ppm
I20260806 13:00:29.599618 10460 webserver.cc:460] Webserver started at http://127.24.83.1:33287/ using document root <none> and password file <none>
I20260806 13:00:29.599858 10460 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:29.599960 10460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:29.600112 10460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:29.601296 10460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "adc24ffbd21c48d780d8b8875b9e78fd"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "246a2494e60ac55b4e1a52cedf96f63d"
server_key_iv: "c1e4bed61183638f903bfadb84869923"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.601665 10460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "adc24ffbd21c48d780d8b8875b9e78fd"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "246a2494e60ac55b4e1a52cedf96f63d"
server_key_iv: "c1e4bed61183638f903bfadb84869923"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.603027 10460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:00:29.603828 10474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.604017 10460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:00:29.604113 10460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "adc24ffbd21c48d780d8b8875b9e78fd"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "246a2494e60ac55b4e1a52cedf96f63d"
server_key_iv: "c1e4bed61183638f903bfadb84869923"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.604197 10460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:29.620157 10460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:29.620458 10460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:29.620610 10460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:29.620865 10460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:29.621191 10460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:29.621244 10460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.621279 10460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:29.621323 10460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.626922 10460 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:34773
I20260806 13:00:29.627014 10592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:34773 every 8 connection(s)
I20260806 13:00:29.627317 10460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 13:00:29.631976 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 10460
I20260806 13:00:29.632103 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 13:00:29.632305 24908 external_mini_cluster.cc:1428] Setting key 0e400ebecc20ef71643078e4f5bcdc17
I20260806 13:00:29.632849 10457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.628081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:37651 (local address 127.24.83.62:40971)
0806 13:00:29.628151 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.628153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.628364 (+ 211us) server_negotiation.cc:407] Connection header received
0806 13:00:29.628885 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.628887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.628903 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.628966 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:29.629681 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.630184 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.631171 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.631375 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.631452 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.632494 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:29.632496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:29.632496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:29.632502 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:29.632524 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:29.632531 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:29.632668 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.632764 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.632791 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 13:00:29.633109 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.628061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40971 (local address 127.24.83.1:37651)
0806 13:00:29.628236 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.628276 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.628780 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.629039 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.629043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.629053 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:29.629585 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.629589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.630301 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.630302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.631075 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.631078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.631474 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.631475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.631608 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.632261 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:00:29.632261 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:29.632290 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:00:29.632313 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:29.632673 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:29.632681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:29.632684 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.632760 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.632864 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260806 13:00:29.633495 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40971
--builtin_ntp_servers=127.24.83.20:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 13:00:29.633558 10593 heartbeater.cc:347] Connected to a master server at 127.24.83.62:40971
I20260806 13:00:29.633648 10593 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:29.633848 10593 heartbeater.cc:511] Master 127.24.83.62:40971 requested a full tablet report, sending...
I20260806 13:00:29.634297 10399 ts_manager.cc:194] Registered new tserver with Master: adc24ffbd21c48d780d8b8875b9e78fd (127.24.83.1:34773)
I20260806 13:00:29.634914 10399 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.1:37651
W20260806 13:00:29.715272 10596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:29.715458 10596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:29.715479 10596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:29.716622 10596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:29.716661 10596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:29.716686 10596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 13:00:29.718659 10596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:29.718916 10596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:29.719233 10596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:29.724761 10604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:29.724769 10601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:29.725174 10596 server_base.cc:1034] running on GCE node
W20260806 13:00:29.725698 10602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:29.725977 10596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:29.726243 10596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:29.727418 10596 hybrid_clock.cc:648] HybridClock initialized: now 1786021229727398 us; error 33 us; skew 500 ppm
I20260806 13:00:29.728852 10596 webserver.cc:460] Webserver started at http://127.24.83.2:34509/ using document root <none> and password file <none>
I20260806 13:00:29.729141 10596 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:29.729231 10596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:29.729394 10596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:29.730702 10596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "35b6e32419d941ce8aba637a71111e93"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "56089e3bb6f53ddc710f0e528a6c8e37"
server_key_iv: "0d082c2f1c59417ecd22b18f60e0a668"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.731181 10596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "35b6e32419d941ce8aba637a71111e93"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "56089e3bb6f53ddc710f0e528a6c8e37"
server_key_iv: "0d082c2f1c59417ecd22b18f60e0a668"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.732980 10596 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 13:00:29.733942 10610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.734092 10596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:00:29.734194 10596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "35b6e32419d941ce8aba637a71111e93"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "56089e3bb6f53ddc710f0e528a6c8e37"
server_key_iv: "0d082c2f1c59417ecd22b18f60e0a668"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.734269 10596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:29.745997 10596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:29.746271 10596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:29.746415 10596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:29.746621 10596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:29.746943 10596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:29.746975 10596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.747015 10596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:29.747042 10596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.752904 10596 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:37685
I20260806 13:00:29.752969 10727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:37685 every 8 connection(s)
I20260806 13:00:29.753293 10596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 13:00:29.758365 10457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.753956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:57661 (local address 127.24.83.62:40971)
0806 13:00:29.754040 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.754043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.754215 (+ 172us) server_negotiation.cc:407] Connection header received
0806 13:00:29.754722 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.754724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.754736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.754788 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:29.755437 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.756007 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.756955 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.757223 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.757310 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.758074 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:29.758076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:29.758076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:29.758081 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:29.758096 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:29.758101 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:29.758151 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.758253 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.758287 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 13:00:29.758569 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.753921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40971 (local address 127.24.83.2:57661)
0806 13:00:29.754100 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.754130 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.754609 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.754858 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.754862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.754870 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:29.755345 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.755349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.756097 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.756098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.756863 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.756865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.757325 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.757326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.757446 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.757943 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:00:29.757943 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:29.757969 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:00:29.757990 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:29.758158 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:29.758163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:29.758166 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.758242 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.758342 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
I20260806 13:00:29.758780 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 10596
I20260806 13:00:29.758889 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 13:00:29.759109 24908 external_mini_cluster.cc:1428] Setting key 7c22b4119cdf17f65b252478a046a41d
I20260806 13:00:29.759239 10728 heartbeater.cc:347] Connected to a master server at 127.24.83.62:40971
I20260806 13:00:29.759327 10728 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:29.759497 10728 heartbeater.cc:511] Master 127.24.83.62:40971 requested a full tablet report, sending...
I20260806 13:00:29.759943 10399 ts_manager.cc:194] Registered new tserver with Master: 35b6e32419d941ce8aba637a71111e93 (127.24.83.2:37685)
I20260806 13:00:29.760123 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40971
--builtin_ntp_servers=127.24.83.20:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 13:00:29.760406 10399 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.2:57661
W20260806 13:00:29.841588 10731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:29.841815 10731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:29.841871 10731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:29.842980 10731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:29.843045 10731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:29.843097 10731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 13:00:29.844570 10731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:40971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:29.844822 10731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:29.845109 10731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:29.849836 10737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:29.849887 10739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:29.849838 10736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:29.850066 10731 server_base.cc:1034] running on GCE node
I20260806 13:00:29.850281 10731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:29.850468 10731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:29.851594 10731 hybrid_clock.cc:648] HybridClock initialized: now 1786021229851586 us; error 25 us; skew 500 ppm
I20260806 13:00:29.852908 10731 webserver.cc:460] Webserver started at http://127.24.83.3:44203/ using document root <none> and password file <none>
I20260806 13:00:29.853145 10731 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:29.853193 10731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:29.853340 10731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:29.854401 10731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "852b521130fe31be34a1f9c285b6c116"
server_key_iv: "193bd1b6b6989c265209907e2544ede8"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.854768 10731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "852b521130fe31be34a1f9c285b6c116"
server_key_iv: "193bd1b6b6989c265209907e2544ede8"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.856062 10731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:00:29.856873 10745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.857043 10731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:29.857138 10731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e"
format_stamp: "Formatted at 2026-08-06 13:00:29 on dist-test-slave-zkpd"
server_key: "852b521130fe31be34a1f9c285b6c116"
server_key_iv: "193bd1b6b6989c265209907e2544ede8"
server_key_version: "encryptionkey@0"
I20260806 13:00:29.857224 10731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:29.864349 10731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:29.864646 10731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:29.864802 10731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:29.865005 10731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:29.865312 10731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:29.865370 10731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.865417 10731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:29.865445 10731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:29.870980 10731 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:32953
I20260806 13:00:29.871022 10861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:32953 every 8 connection(s)
I20260806 13:00:29.871305 10731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 13:00:29.874805 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 10731
I20260806 13:00:29.874925 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 13:00:29.875113 24908 external_mini_cluster.cc:1428] Setting key af01783b1ad41b941e8bd3e8af9ceb3c
I20260806 13:00:29.876410 10457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.871973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:54831 (local address 127.24.83.62:40971)
0806 13:00:29.872044 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.872046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.872291 (+ 245us) server_negotiation.cc:407] Connection header received
0806 13:00:29.872716 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.872717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.872727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.872789 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:29.873370 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.873868 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.874901 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.875165 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.875226 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.876110 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:29.876111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:29.876111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:29.876116 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:29.876132 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:29.876136 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:29.876185 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.876302 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.876331 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 13:00:29.876616 10864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.871984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:40971 (local address 127.24.83.3:54831)
0806 13:00:29.872183 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.872213 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.872637 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.872839 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.872841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.872849 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:29.873304 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.873308 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.873943 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.873944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.874790 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.874793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.875248 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.875249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.875379 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.875959 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:00:29.875960 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:29.875987 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:00:29.876007 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:29.876195 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:29.876200 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:29.876203 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.876301 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.876400 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260806 13:00:29.877084 10862 heartbeater.cc:347] Connected to a master server at 127.24.83.62:40971
I20260806 13:00:29.877157 10862 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:29.877293 10862 heartbeater.cc:511] Master 127.24.83.62:40971 requested a full tablet report, sending...
I20260806 13:00:29.877609 10399 ts_manager.cc:194] Registered new tserver with Master: 8dd6f9e0033f49bc95e638d8bdb7d52e (127.24.83.3:32953)
I20260806 13:00:29.877981 10399 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.3:54831
I20260806 13:00:29.886687 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:00:29.890646 10457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.888051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34430 (local address 127.24.83.62:40971)
0806 13:00:29.888116 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.888118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.888202 (+ 84us) server_negotiation.cc:407] Connection header received
0806 13:00:29.888252 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.888253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.888261 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.888312 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:29.888477 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.888924 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.889591 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.889758 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.889805 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.890464 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:29.890465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:29.890465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:29.890469 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:29.890478 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:29.890481 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:29.890517 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.890577 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.890596 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 13:00:29.892088 10399 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:00:29.893589 10399 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:00:29.899384 10871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.895640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:34773 (local address 127.0.0.1:47944)
0806 13:00:29.895814 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.895853 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.895950 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.896180 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.896183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.896185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:29.896250 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.896252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.897428 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.897429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.898284 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.898285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.898997 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.898998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.899072 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.899155 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.899243 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.899301 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":60,"thread_start_us":67,"threads_started":1}
I20260806 13:00:29.899402 10872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.895726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.24.83.1:34773)
0806 13:00:29.895943 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.895948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.895961 (+ 13us) server_negotiation.cc:407] Connection header received
0806 13:00:29.896060 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.896062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.896077 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.896166 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:29.896342 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.897332 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.898367 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.898922 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.898980 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.899039 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.899264 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.899303 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260806 13:00:29.900213 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.895643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:37685 (local address 127.0.0.1:44598)
0806 13:00:29.896166 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.896176 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.896219 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.896846 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.896847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.896848 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:29.896879 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.896880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.898203 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.898204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.899047 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.899049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.899966 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.899967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.900033 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.900037 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.900099 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.900147 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":81,"threads_started":1}
I20260806 13:00:29.900270 10874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.895944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44598 (local address 127.24.83.2:37685)
0806 13:00:29.896695 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.896699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.896708 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:00:29.896747 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.896749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.896763 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.896840 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:29.896965 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.898109 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.899136 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.899845 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.899961 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.900041 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.900111 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.900154 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":112,"threads_started":1}
I20260806 13:00:29.900432 10526 tablet_service.cc:1467] Processing CreateTablet for tablet a74221289d2a45a0b541190b42a664ae (DEFAULT_TABLE table=test-table [id=68f24b1776564aef8ce9eb80f0821ad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:29.900780 10526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a74221289d2a45a0b541190b42a664ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:29.900914 10661 tablet_service.cc:1467] Processing CreateTablet for tablet a74221289d2a45a0b541190b42a664ae (DEFAULT_TABLE table=test-table [id=68f24b1776564aef8ce9eb80f0821ad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:29.901167 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a74221289d2a45a0b541190b42a664ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:29.903297 10876 tablet_bootstrap.cc:492] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd: Bootstrap starting.
I20260806 13:00:29.903748 10877 tablet_bootstrap.cc:492] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93: Bootstrap starting.
I20260806 13:00:29.904153 10876 tablet_bootstrap.cc:654] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:29.904379 10877 tablet_bootstrap.cc:654] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:29.904523 10876 log.cc:824] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:29.904597 10875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.896107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42520 (local address 127.24.83.3:32953)
0806 13:00:29.896268 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.896272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.896442 (+ 170us) server_negotiation.cc:407] Connection header received
0806 13:00:29.896481 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.896482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.896497 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.896561 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:29.897025 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.898094 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.900253 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.904148 (+ 3895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.904229 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.904320 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.904467 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.904511 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260806 13:00:29.904733 10877 log.cc:824] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:29.904819 10873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.895867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:32953 (local address 127.0.0.1:42520)
0806 13:00:29.896324 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.896335 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.896387 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.896919 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.896920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.896923 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:29.896964 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.896966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.898198 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.898199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.900164 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.900167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.904296 (+ 4129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.904297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.904369 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.904374 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.904490 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.904547 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":748,"thread_start_us":124,"threads_started":1}
I20260806 13:00:29.905489 10876 tablet_bootstrap.cc:492] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd: No bootstrap required, opened a new log
I20260806 13:00:29.905555 10877 tablet_bootstrap.cc:492] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93: No bootstrap required, opened a new log
I20260806 13:00:29.905565 10876 ts_tablet_manager.cc:1397] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:29.905623 10877 ts_tablet_manager.cc:1397] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:29.905539 10795 tablet_service.cc:1467] Processing CreateTablet for tablet a74221289d2a45a0b541190b42a664ae (DEFAULT_TABLE table=test-table [id=68f24b1776564aef8ce9eb80f0821ad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:29.905848 10795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a74221289d2a45a0b541190b42a664ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:29.907752 10877 raft_consensus.cc:348] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34773 } } peers { permanent_uuid: "35b6e32419d941ce8aba637a71111e93" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37685 } } peers { permanent_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 32953 } }
I20260806 13:00:29.907925 10877 raft_consensus.cc:374] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:29.907974 10877 raft_consensus.cc:729] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b6e32419d941ce8aba637a71111e93, State: Initialized, Role: FOLLOWER
I20260806 13:00:29.908114 10877 consensus_queue.cc:260] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34773 } } peers { permanent_uuid: "35b6e32419d941ce8aba637a71111e93" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37685 } } peers { permanent_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 32953 } }
I20260806 13:00:29.908144 10876 raft_consensus.cc:348] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34773 } } peers { permanent_uuid: "35b6e32419d941ce8aba637a71111e93" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37685 } } peers { permanent_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 32953 } }
I20260806 13:00:29.908301 10876 raft_consensus.cc:374] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:29.908344 10876 raft_consensus.cc:729] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc24ffbd21c48d780d8b8875b9e78fd, State: Initialized, Role: FOLLOWER
I20260806 13:00:29.908385 10877 ts_tablet_manager.cc:1428] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:00:29.908450 10876 consensus_queue.cc:260] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34773 } } peers { permanent_uuid: "35b6e32419d941ce8aba637a71111e93" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37685 } } peers { permanent_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 32953 } }
I20260806 13:00:29.908519 10728 heartbeater.cc:503] Master 127.24.83.62:40971 was elected leader, sending a full tablet report...
I20260806 13:00:29.908695 10876 ts_tablet_manager.cc:1428] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:00:29.908746 10593 heartbeater.cc:503] Master 127.24.83.62:40971 was elected leader, sending a full tablet report...
I20260806 13:00:29.909724 10882 tablet_bootstrap.cc:492] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e: Bootstrap starting.
I20260806 13:00:29.910282 10882 tablet_bootstrap.cc:654] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:29.910560 10882 log.cc:824] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:29.911293 10882 tablet_bootstrap.cc:492] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e: No bootstrap required, opened a new log
I20260806 13:00:29.911365 10882 ts_tablet_manager.cc:1397] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:29.913158 10882 raft_consensus.cc:348] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34773 } } peers { permanent_uuid: "35b6e32419d941ce8aba637a71111e93" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37685 } } peers { permanent_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 32953 } }
I20260806 13:00:29.913303 10882 raft_consensus.cc:374] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:29.913324 10882 raft_consensus.cc:729] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd6f9e0033f49bc95e638d8bdb7d52e, State: Initialized, Role: FOLLOWER
I20260806 13:00:29.913467 10882 consensus_queue.cc:260] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34773 } } peers { permanent_uuid: "35b6e32419d941ce8aba637a71111e93" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37685 } } peers { permanent_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 32953 } }
I20260806 13:00:29.913736 10882 ts_tablet_manager.cc:1428] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:00:29.913777 10862 heartbeater.cc:503] Master 127.24.83.62:40971 was elected leader, sending a full tablet report...
I20260806 13:00:29.920902 10880 raft_consensus.cc:482] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:00:29.921017 10880 raft_consensus.cc:504] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34773 } } peers { permanent_uuid: "35b6e32419d941ce8aba637a71111e93" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37685 } } peers { permanent_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 32953 } }
I20260806 13:00:29.921317 10880 leader_election.cc:288] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adc24ffbd21c48d780d8b8875b9e78fd (127.24.83.1:34773), 8dd6f9e0033f49bc95e638d8bdb7d52e (127.24.83.3:32953)
I20260806 13:00:29.924479 10872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.921547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:39055 (local address 127.24.83.1:34773)
0806 13:00:29.921602 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.921604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.921615 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:00:29.921641 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.921642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.921652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.921703 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:29.921870 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.922618 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.923661 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.924134 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.924190 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.924198 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.924359 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.924412 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 13:00:29.924468 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.921465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:34773 (local address 127.24.83.2:39055)
0806 13:00:29.921527 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.921539 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.921591 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.921725 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.921726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.921728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:29.921793 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.921794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.922700 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.922701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.923575 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.923577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.924215 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.924216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.924288 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.924291 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.924341 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.924379 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 13:00:29.924544 10875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.921634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:54487 (local address 127.24.83.3:32953)
0806 13:00:29.921681 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.921683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.921780 (+ 97us) server_negotiation.cc:407] Connection header received
0806 13:00:29.921852 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.921853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.921862 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.921917 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:29.922047 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.922751 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.923748 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.924182 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.924253 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.924272 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.924442 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.924478 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 13:00:29.924641 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.921580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:32953 (local address 127.24.83.2:54487)
0806 13:00:29.921706 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.921716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:29.921791 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:29.921934 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:29.921935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:29.921936 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:29.921986 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.921987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.922820 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.922821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.923667 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:29.923669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.924272 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.924273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:29.924367 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.924370 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:29.924425 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 13:00:29.924461 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260806 13:00:29.924839 10547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74221289d2a45a0b541190b42a664ae" candidate_uuid: "35b6e32419d941ce8aba637a71111e93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" is_pre_election: true
I20260806 13:00:29.924919 10815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74221289d2a45a0b541190b42a664ae" candidate_uuid: "35b6e32419d941ce8aba637a71111e93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" is_pre_election: true
I20260806 13:00:29.924989 10547 raft_consensus.cc:2454] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b6e32419d941ce8aba637a71111e93 in term 0.
I20260806 13:00:29.925062 10815 raft_consensus.cc:2454] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b6e32419d941ce8aba637a71111e93 in term 0.
I20260806 13:00:29.925220 10613 leader_election.cc:302] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b6e32419d941ce8aba637a71111e93, adc24ffbd21c48d780d8b8875b9e78fd; no voters:
I20260806 13:00:29.925370 10880 raft_consensus.cc:2781] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:00:29.925444 10880 raft_consensus.cc:482] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:00:29.925462 10880 raft_consensus.cc:3037] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:29.926123 10880 raft_consensus.cc:504] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34773 } } peers { permanent_uuid: "35b6e32419d941ce8aba637a71111e93" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37685 } } peers { permanent_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 32953 } }
I20260806 13:00:29.926246 10880 leader_election.cc:288] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [CANDIDATE]: Term 1 election: Requested vote from peers adc24ffbd21c48d780d8b8875b9e78fd (127.24.83.1:34773), 8dd6f9e0033f49bc95e638d8bdb7d52e (127.24.83.3:32953)
I20260806 13:00:29.926417 10547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74221289d2a45a0b541190b42a664ae" candidate_uuid: "35b6e32419d941ce8aba637a71111e93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc24ffbd21c48d780d8b8875b9e78fd"
I20260806 13:00:29.926435 10815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74221289d2a45a0b541190b42a664ae" candidate_uuid: "35b6e32419d941ce8aba637a71111e93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e"
I20260806 13:00:29.926501 10815 raft_consensus.cc:3037] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:29.926498 10547 raft_consensus.cc:3037] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:29.927174 10815 raft_consensus.cc:2454] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b6e32419d941ce8aba637a71111e93 in term 1.
I20260806 13:00:29.927204 10547 raft_consensus.cc:2454] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b6e32419d941ce8aba637a71111e93 in term 1.
I20260806 13:00:29.927371 10613 leader_election.cc:302] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b6e32419d941ce8aba637a71111e93, adc24ffbd21c48d780d8b8875b9e78fd; no voters:
I20260806 13:00:29.927479 10880 raft_consensus.cc:2781] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:29.927614 10880 raft_consensus.cc:686] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 1 LEADER]: Becoming Leader. State: Replica: 35b6e32419d941ce8aba637a71111e93, State: Running, Role: LEADER
I20260806 13:00:29.927724 10880 consensus_queue.cc:237] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34773 } } peers { permanent_uuid: "35b6e32419d941ce8aba637a71111e93" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37685 } } peers { permanent_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 32953 } }
I20260806 13:00:29.928499 10398 catalog_manager.cc:5495] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35b6e32419d941ce8aba637a71111e93 (127.24.83.2). New cstate: current_term: 1 leader_uuid: "35b6e32419d941ce8aba637a71111e93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35b6e32419d941ce8aba637a71111e93" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 37685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 32953 } health_report { overall_health: UNKNOWN } } }
I20260806 13:00:29.934285 10874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.932321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.24.83.2:37685)
0806 13:00:29.932373 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:29.932375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:29.932391 (+ 16us) server_negotiation.cc:407] Connection header received
0806 13:00:29.932439 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:29.932439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:29.932453 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:29.932502 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:00:29.932576 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.932997 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.933666 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:29.933854 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:29.933904 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:29.934035 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:00:29.934118 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:00:29.934176 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:29.934207 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 13:00:29.934221 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 13:00:29.935796 10815 raft_consensus.cc:1261] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e [term 1 FOLLOWER]: Refusing update from remote peer 35b6e32419d941ce8aba637a71111e93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:29.935871 10547 raft_consensus.cc:1261] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd [term 1 FOLLOWER]: Refusing update from remote peer 35b6e32419d941ce8aba637a71111e93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:29.936124 10880 consensus_queue.cc:1035] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dd6f9e0033f49bc95e638d8bdb7d52e" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 32953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:29.936280 10886 consensus_queue.cc:1035] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adc24ffbd21c48d780d8b8875b9e78fd" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:29.938681 10891 mvcc.cc:205] Tried to move back new op lower bound from 7315542957814886400 to 7315542957784772608. Current Snapshot: MvccSnapshot[applied={T|T < 7315542957814886400}]
I20260806 13:00:29.939162 10890 mvcc.cc:205] Tried to move back new op lower bound from 7315542957814886400 to 7315542957784772608. Current Snapshot: MvccSnapshot[applied={T|T < 7315542957814886400}]
I20260806 13:00:29.940666 10399 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:29.940788 10399 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:29.941569 10399 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61ceea18ebdf4679b0238c6de23b1305: Sending DeleteTablet for 3 replicas of tablet a74221289d2a45a0b541190b42a664ae
I20260806 13:00:29.941890 10661 tablet_service.cc:1514] Processing DeleteTablet for tablet a74221289d2a45a0b541190b42a664ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:29 UTC) from {username='slave'} at 127.0.0.1:44598
I20260806 13:00:29.941942 10795 tablet_service.cc:1514] Processing DeleteTablet for tablet a74221289d2a45a0b541190b42a664ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:29 UTC) from {username='slave'} at 127.0.0.1:42520
I20260806 13:00:29.941948 10526 tablet_service.cc:1514] Processing DeleteTablet for tablet a74221289d2a45a0b541190b42a664ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:29 UTC) from {username='slave'} at 127.0.0.1:47944
I20260806 13:00:29.942164 10901 tablet_replica.cc:330] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd: stopping tablet replica
I20260806 13:00:29.942273 10901 raft_consensus.cc:2229] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:00:29.942368 10899 tablet_replica.cc:330] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93: stopping tablet replica
I20260806 13:00:29.942387 10901 raft_consensus.cc:2258] T a74221289d2a45a0b541190b42a664ae P adc24ffbd21c48d780d8b8875b9e78fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:29.942473 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 10460
I20260806 13:00:29.945027 10899 raft_consensus.cc:2229] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:00:29.945176 10899 raft_consensus.cc:2258] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:29.945433 10900 tablet_replica.cc:330] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e: stopping tablet replica
I20260806 13:00:29.945568 10900 raft_consensus.cc:2229] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:00:29.945683 10900 raft_consensus.cc:2258] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:29.945693 10899 ts_tablet_manager.cc:1905] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:00:29.946103 10900 ts_tablet_manager.cc:1905] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:00:29.947427 10899 ts_tablet_manager.cc:1918] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:00:29.947506 10899 log.cc:1196] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/a74221289d2a45a0b541190b42a664ae
I20260806 13:00:29.947738 10900 ts_tablet_manager.cc:1918] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:00:29.947784 10899 ts_tablet_manager.cc:1939] T a74221289d2a45a0b541190b42a664ae P 35b6e32419d941ce8aba637a71111e93: Deleting consensus metadata
I20260806 13:00:29.947815 10900 log.cc:1196] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/a74221289d2a45a0b541190b42a664ae
I20260806 13:00:29.948068 10900 ts_tablet_manager.cc:1939] T a74221289d2a45a0b541190b42a664ae P 8dd6f9e0033f49bc95e638d8bdb7d52e: Deleting consensus metadata
I20260806 13:00:29.948437 10383 catalog_manager.cc:4841] TS 35b6e32419d941ce8aba637a71111e93 (127.24.83.2:37685): tablet a74221289d2a45a0b541190b42a664ae (table test-table [id=68f24b1776564aef8ce9eb80f0821ad9]) successfully deleted
I20260806 13:00:29.948457 10385 catalog_manager.cc:4841] TS 8dd6f9e0033f49bc95e638d8bdb7d52e (127.24.83.3:32953): tablet a74221289d2a45a0b541190b42a664ae (table test-table [id=68f24b1776564aef8ce9eb80f0821ad9]) successfully deleted
W20260806 13:00:29.951408 10385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:34773 (error 108)
I20260806 13:00:29.951622 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 10596
I20260806 13:00:29.953390 10873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:29.953249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:34773 (local address 127.0.0.1:47960)
0806 13:00:29.953311 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:29.953355 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:34773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 13:00:29.953580 10385 catalog_manager.cc:4584] TS adc24ffbd21c48d780d8b8875b9e78fd (127.24.83.1:34773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a74221289d2a45a0b541190b42a664ae: Network error: Client connection negotiation failed: client connection to 127.24.83.1:34773: connect: Connection refused (error 111)
I20260806 13:00:29.960153 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 10731
I20260806 13:00:29.969118 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 10368
2026-08-06T13:00:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (636 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:30 dist-test-slave-zkpd krb5kdc[10905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:30 dist-test-slave-zkpd krb5kdc[10905](info): set up 2 sockets
Aug 06 13:00:30 dist-test-slave-zkpd krb5kdc[10905](info): commencing operation
krb5kdc: starting...
W20260806 13:00:31.011782 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:31 dist-test-slave-zkpd krb5kdc[10905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 13:00:31.127518 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:36109
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:36109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:31.209751 10920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:31.209945 10920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:31.209966 10920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:31.211018 10920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:31.211052 10920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:31.211066 10920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:31.211079 10920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:31.211091 10920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:31.212605 10920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:36109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:36109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:31.212795 10920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:31.212955 10920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:31.217612 10928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:31.217693 10925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:31.217747 10926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:31.217986 10920 server_base.cc:1034] running on GCE node
I20260806 13:00:31.218223 10920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:31.218509 10920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:31.219663 10920 hybrid_clock.cc:648] HybridClock initialized: now 1786021231219639 us; error 37 us; skew 500 ppm
Aug 06 13:00:31 dist-test-slave-zkpd krb5kdc[10905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:31.222210 10920 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:00:31.222671 10920 webserver.cc:460] Webserver started at http://127.24.83.62:44871/ using document root <none> and password file <none>
I20260806 13:00:31.222914 10920 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:31.222983 10920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:31.223134 10920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:31.224236 10920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "a5ca26dcc1d5414d9776ded3ffe0b966"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "ba80cbba578440c6aabee4755cd0f5e7"
server_key_iv: "5f729743dba5cb98eca0ed7ffa7e03d8"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.224588 10920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "a5ca26dcc1d5414d9776ded3ffe0b966"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "ba80cbba578440c6aabee4755cd0f5e7"
server_key_iv: "5f729743dba5cb98eca0ed7ffa7e03d8"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.225916 10920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:00:31.226645 10935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:31.226815 10920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:31.226924 10920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "a5ca26dcc1d5414d9776ded3ffe0b966"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "ba80cbba578440c6aabee4755cd0f5e7"
server_key_iv: "5f729743dba5cb98eca0ed7ffa7e03d8"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.227000 10920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:31.244553 10920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:31.246984 10920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:31.247153 10920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:31.251319 10920 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:36109
I20260806 13:00:31.251356 10990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:36109 every 8 connection(s)
I20260806 13:00:31.251802 10920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:31.252338 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 10920
I20260806 13:00:31.252475 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:31.252714 24908 external_mini_cluster.cc:1428] Setting key 90aae1907dae6aec8094ce5f76fadfcd
I20260806 13:00:31.252930 10991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:31.256179 10991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966: Bootstrap starting.
I20260806 13:00:31.256870 10991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:31.257177 10991 log.cc:824] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:31.257848 10991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966: No bootstrap required, opened a new log
Aug 06 13:00:31 dist-test-slave-zkpd krb5kdc[10905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:31.260434 10991 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca26dcc1d5414d9776ded3ffe0b966" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 36109 } }
I20260806 13:00:31.260622 10991 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:31.260663 10991 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5ca26dcc1d5414d9776ded3ffe0b966, State: Initialized, Role: FOLLOWER
I20260806 13:00:31.260807 10991 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca26dcc1d5414d9776ded3ffe0b966" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 36109 } }
I20260806 13:00:31.260897 10991 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:31.260936 10991 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:31.260986 10991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:31.261488 10991 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca26dcc1d5414d9776ded3ffe0b966" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 36109 } }
I20260806 13:00:31.261607 10991 leader_election.cc:302] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5ca26dcc1d5414d9776ded3ffe0b966; no voters:
I20260806 13:00:31.261760 10991 leader_election.cc:288] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:31.261806 10996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:31.261943 10996 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [term 1 LEADER]: Becoming Leader. State: Replica: a5ca26dcc1d5414d9776ded3ffe0b966, State: Running, Role: LEADER
I20260806 13:00:31.262019 10991 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:31.262027 10996 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca26dcc1d5414d9776ded3ffe0b966" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 36109 } }
I20260806 13:00:31.262362 10996 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5ca26dcc1d5414d9776ded3ffe0b966. Latest consensus state: current_term: 1 leader_uuid: "a5ca26dcc1d5414d9776ded3ffe0b966" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca26dcc1d5414d9776ded3ffe0b966" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 36109 } } }
I20260806 13:00:31.262447 10996 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:31.262382 10999 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5ca26dcc1d5414d9776ded3ffe0b966" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca26dcc1d5414d9776ded3ffe0b966" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 36109 } } }
I20260806 13:00:31.262626 10999 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:31.262658 11003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:31.263254 11003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:31.264096 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.253274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40158 (local address 127.24.83.62:36109)
0806 13:00:31.253468 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.253477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.253512 (+ 35us) server_negotiation.cc:407] Connection header received
0806 13:00:31.254074 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.254081 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.254129 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.254245 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:31.255015 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.255872 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.256977 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.257441 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.257559 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.260141 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:31.260146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:31.260146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:31.260157 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:31.262020 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:31.262421 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:31.262424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:31.262424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:31.262531 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:31.262764 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:31.262765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:31.262765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:31.263082 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:31.263654 (+ 572us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.263723 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.263859 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
I20260806 13:00:31.265197 11003 catalog_manager.cc:1347] Generated new cluster ID: 59338cfb920d42c09a7842c1f6469832
I20260806 13:00:31.265249 11003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:31.269647 11003 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:31.270337 11003 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:31.288579 11003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966: Generated new TSK 0
I20260806 13:00:31.288764 11003 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 13:00:31.316272 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:36109
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:31.398507 11018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:31.398741 11018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:31.398780 11018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:31.400035 11018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:31.400098 11018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:31.400143 11018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 13:00:31.401597 11018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:36109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:31.401858 11018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:31.402088 11018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:31.407114 11024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:31.407116 11027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:31.407107 11025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:31.407411 11018 server_base.cc:1034] running on GCE node
I20260806 13:00:31.407600 11018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:31.407804 11018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:31.408957 11018 hybrid_clock.cc:648] HybridClock initialized: now 1786021231408949 us; error 42 us; skew 500 ppm
Aug 06 13:00:31 dist-test-slave-zkpd krb5kdc[10905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:31.411506 11018 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 13:00:31.411958 11018 webserver.cc:460] Webserver started at http://127.24.83.1:33191/ using document root <none> and password file <none>
I20260806 13:00:31.412137 11018 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:31.412177 11018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:31.412261 11018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:31.413196 11018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "7ead6d42f73246d59fd03f1d9025945b"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "da2992c7702c1136d7d93719b5b075a9"
server_key_iv: "5b3c37c962e68cf7312eb309b5dba020"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.413478 11018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "7ead6d42f73246d59fd03f1d9025945b"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "da2992c7702c1136d7d93719b5b075a9"
server_key_iv: "5b3c37c962e68cf7312eb309b5dba020"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.414773 11018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:00:31.415413 11034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:31.415573 11018 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:31.415658 11018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "7ead6d42f73246d59fd03f1d9025945b"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "da2992c7702c1136d7d93719b5b075a9"
server_key_iv: "5b3c37c962e68cf7312eb309b5dba020"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.415714 11018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:31.430604 11018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:31.432946 11018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:31.433056 11018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:31.433231 11018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:31.433563 11018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:31.433599 11018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:31.433620 11018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:31.433668 11018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:31.439242 11018 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:43659
I20260806 13:00:31.439320 11151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:43659 every 8 connection(s)
I20260806 13:00:31.439638 11018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 13:00:31.441249 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 11018
I20260806 13:00:31.441375 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 13:00:31.441574 24908 external_mini_cluster.cc:1428] Setting key f003b8ed5a063b1cfdf31d339f9a5f83
Aug 06 13:00:31 dist-test-slave-zkpd krb5kdc[10905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:31.450429 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.440416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:36381 (local address 127.24.83.62:36109)
0806 13:00:31.440474 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.440476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.440672 (+ 196us) server_negotiation.cc:407] Connection header received
0806 13:00:31.441765 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.441766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.441776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.441860 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:31.443088 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.443582 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.444684 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.445038 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.445118 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.447592 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:31.447594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:31.447594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:31.447600 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:31.448980 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:31.449473 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:31.449473 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:31.449474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:31.449522 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:31.449803 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:31.449804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:31.449804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:31.449906 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:31.449955 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.450299 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.450364 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 13:00:31.450755 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.440381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:36109 (local address 127.24.83.1:36381)
0806 13:00:31.440553 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:31.440594 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:31.441062 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:31.441943 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:31.441948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:31.442713 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:31.442985 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.442992 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.443690 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.443691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.444592 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.444595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.445128 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.445129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.445224 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.445708 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:00:31.445715 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:31.447494 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:31.449068 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:31.449076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:31.449080 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:31.449394 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:31.449589 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:31.449590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:31.449591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:31.449735 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:31.449976 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:31.449983 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:31.450158 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 13:00:31.450290 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 13:00:31.450473 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
I20260806 13:00:31.451333 11152 heartbeater.cc:347] Connected to a master server at 127.24.83.62:36109
I20260806 13:00:31.451431 11152 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:31.451565 11152 heartbeater.cc:511] Master 127.24.83.62:36109 requested a full tablet report, sending...
I20260806 13:00:31.452106 10953 ts_manager.cc:194] Registered new tserver with Master: 7ead6d42f73246d59fd03f1d9025945b (127.24.83.1:43659)
I20260806 13:00:31.452669 10953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:36381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 13:00:31.472676 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:36109
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:31.556272 11157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:31.556466 11157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:31.556489 11157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:31.557607 11157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:31.557649 11157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:31.557701 11157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 13:00:31.559125 11157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:36109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:31.559353 11157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:31.559576 11157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:31.564553 11165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:31.564549 11162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:31.564584 11163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:31.564766 11157 server_base.cc:1034] running on GCE node
I20260806 13:00:31.565011 11157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:31.565269 11157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:31.566414 11157 hybrid_clock.cc:648] HybridClock initialized: now 1786021231566396 us; error 40 us; skew 500 ppm
Aug 06 13:00:31 dist-test-slave-zkpd krb5kdc[10905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:31.569044 11157 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 13:00:31.569502 11157 webserver.cc:460] Webserver started at http://127.24.83.2:40359/ using document root <none> and password file <none>
I20260806 13:00:31.569718 11157 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:31.569762 11157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:31.569931 11157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:31.570911 11157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "aa103e644ada44c68bc53033bc1649e5"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "fdc6525954d38c378cf0a8ee2d8f44ac"
server_key_iv: "c921a54d84bbbcb9e7accc4adf9073e3"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.571281 11157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "aa103e644ada44c68bc53033bc1649e5"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "fdc6525954d38c378cf0a8ee2d8f44ac"
server_key_iv: "c921a54d84bbbcb9e7accc4adf9073e3"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.572711 11157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:00:31.573431 11172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:31.573621 11157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:00:31.573711 11157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "aa103e644ada44c68bc53033bc1649e5"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "fdc6525954d38c378cf0a8ee2d8f44ac"
server_key_iv: "c921a54d84bbbcb9e7accc4adf9073e3"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.573764 11157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:31.596191 11157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:31.597967 11157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:31.598071 11157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:31.598243 11157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:31.598480 11157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:31.598510 11157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:31.598531 11157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:31.598543 11157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:31.604071 11157 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:33167
I20260806 13:00:31.604161 11290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:33167 every 8 connection(s)
I20260806 13:00:31.604449 11157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 13:00:31.607410 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 11157
I20260806 13:00:31.607553 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 13:00:31.607769 24908 external_mini_cluster.cc:1428] Setting key d7ec78737ef9a61da6da82c407a56e86
Aug 06 13:00:31 dist-test-slave-zkpd krb5kdc[10905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:31.614216 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.605151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:41099 (local address 127.24.83.62:36109)
0806 13:00:31.605216 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.605219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.605558 (+ 339us) server_negotiation.cc:407] Connection header received
0806 13:00:31.606002 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.606003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.606014 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.606064 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:31.606910 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.607411 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.608882 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.609073 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.609127 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.611364 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:31.611366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:31.611367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:31.611374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:31.612809 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:31.613214 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:31.613215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:31.613216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:31.613283 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:31.613541 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:31.613542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:31.613542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:31.613662 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:31.613748 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.614036 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.614128 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 13:00:31.614446 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.605129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:36109 (local address 127.24.83.2:41099)
0806 13:00:31.605443 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:31.605478 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:31.605913 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:31.606115 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:31.606118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:31.606658 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:31.606835 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.606840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.607501 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.607503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.608799 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.608803 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.609161 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.609163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.609306 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.609777 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:00:31.609787 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:31.611267 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:31.612909 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:31.612917 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:31.612919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:31.613133 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:31.613351 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:31.613352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:31.613352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:31.613463 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:31.613755 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:31.613763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:31.613895 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 13:00:31.613999 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 13:00:31.614150 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":263,"threads_started":1}
I20260806 13:00:31.615083 11291 heartbeater.cc:347] Connected to a master server at 127.24.83.62:36109
I20260806 13:00:31.615176 11291 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:31.615370 11291 heartbeater.cc:511] Master 127.24.83.62:36109 requested a full tablet report, sending...
I20260806 13:00:31.615739 10953 ts_manager.cc:194] Registered new tserver with Master: aa103e644ada44c68bc53033bc1649e5 (127.24.83.2:33167)
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
I20260806 13:00:31.616196 10953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:41099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 13:00:31.633061 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:36109
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:31.712893 11296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:31.713107 11296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:31.713147 11296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:31.714310 11296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:31.714377 11296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:31.714418 11296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 13:00:31.715919 11296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:36109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:31.716173 11296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:31.716396 11296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:31.721369 11301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:31.721380 11302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:31.721367 11304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:31.721629 11296 server_base.cc:1034] running on GCE node
I20260806 13:00:31.721906 11296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:31.722142 11296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:31.723304 11296 hybrid_clock.cc:648] HybridClock initialized: now 1786021231723285 us; error 30 us; skew 500 ppm
Aug 06 13:00:31 dist-test-slave-zkpd krb5kdc[10905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:31.725739 11296 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 13:00:31.726188 11296 webserver.cc:460] Webserver started at http://127.24.83.3:42749/ using document root <none> and password file <none>
I20260806 13:00:31.726444 11296 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:31.726514 11296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:31.726679 11296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:31.727613 11296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "8d442f1b9cdc48af91d7ad172a4862f0"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "41e4cd16f80b84ee3b2d73c55b75eac1"
server_key_iv: "6cf387ea7e6b69e2eed714d69e3adbe8"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.727959 11296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "8d442f1b9cdc48af91d7ad172a4862f0"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "41e4cd16f80b84ee3b2d73c55b75eac1"
server_key_iv: "6cf387ea7e6b69e2eed714d69e3adbe8"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.729189 11296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:00:31.729944 11311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:31.730137 11296 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:00:31.730247 11296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "8d442f1b9cdc48af91d7ad172a4862f0"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "41e4cd16f80b84ee3b2d73c55b75eac1"
server_key_iv: "6cf387ea7e6b69e2eed714d69e3adbe8"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.730301 11296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:31.742551 11296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:31.744366 11296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:31.744477 11296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:31.744640 11296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:31.744930 11296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:31.744962 11296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:31.744983 11296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:31.745029 11296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:31.750792 11296 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:41519
I20260806 13:00:31.750839 11429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:41519 every 8 connection(s)
I20260806 13:00:31.751113 11296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
Aug 06 13:00:31 dist-test-slave-zkpd krb5kdc[10905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:31.757521 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 11296
I20260806 13:00:31.757664 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 13:00:31.757889 24908 external_mini_cluster.cc:1428] Setting key 6bcee73cd221aec4110759ef715fc0eb
I20260806 13:00:31.759997 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.751782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:54241 (local address 127.24.83.62:36109)
0806 13:00:31.751822 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.751824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.752041 (+ 217us) server_negotiation.cc:407] Connection header received
0806 13:00:31.752491 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.752492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.752501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.752553 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:31.753362 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.753882 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.754806 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.754986 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.755035 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.757000 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:31.757001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:31.757002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:31.757009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:31.758565 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:31.759010 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:31.759012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:31.759012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:31.759073 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:31.759349 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:31.759349 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:31.759350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:31.759464 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:31.759551 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.759808 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.759885 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:00:31.760172 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.751733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:36109 (local address 127.24.83.3:54241)
0806 13:00:31.751884 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:31.751945 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:31.752411 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:31.752600 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:31.752603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:31.753115 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:31.753291 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.753295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.753962 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.753963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.754732 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.754734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.755059 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.755059 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.755152 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.755548 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:00:31.755556 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:31.756894 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:31.758633 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:31.758640 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:31.758642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:31.758938 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:31.759156 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:31.759156 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:31.759157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:31.759274 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:31.759542 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:31.759548 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:31.759680 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 13:00:31.759785 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 13:00:31.759951 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260806 13:00:31.760666 11430 heartbeater.cc:347] Connected to a master server at 127.24.83.62:36109
I20260806 13:00:31.760742 11430 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:31.760936 11430 heartbeater.cc:511] Master 127.24.83.62:36109 requested a full tablet report, sending...
I20260806 13:00:31.761312 10953 ts_manager.cc:194] Registered new tserver with Master: 8d442f1b9cdc48af91d7ad172a4862f0 (127.24.83.3:41519)
I20260806 13:00:31.761678 10953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:54241
I20260806 13:00:31.769649 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:00:31.776618 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.770903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.24.83.62:36109)
0806 13:00:31.770958 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.770965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.771073 (+ 108us) server_negotiation.cc:407] Connection header received
0806 13:00:31.771110 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.771111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.771119 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.771167 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:31.771647 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.772151 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.772815 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.772983 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.773030 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.774198 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:31.774199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:31.774200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:31.774206 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:31.775437 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:31.775800 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:31.775801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:31.775801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:31.775866 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:31.776100 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:31.776101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:31.776101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:31.776275 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:31.776355 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.776463 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.776509 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 13:00:31.778111 10953 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:00:31.779664 10953 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:00:31.785957 11438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.781793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:41519 (local address 127.0.0.1:45616)
0806 13:00:31.782085 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:31.782125 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:31.782196 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:31.782457 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:31.782460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:31.782462 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:31.782537 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.782539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.783850 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.783851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.784994 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.784997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.785650 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.785652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.785739 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.785752 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 13:00:31.785832 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 13:00:31.785887 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":3,"thread_start_us":48,"threads_started":1}
I20260806 13:00:31.786019 11440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.781861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45616 (local address 127.24.83.3:41519)
0806 13:00:31.782029 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.782034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.782180 (+ 146us) server_negotiation.cc:407] Connection header received
0806 13:00:31.782268 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.782270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.782285 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.782362 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:31.782604 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.783747 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.785071 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.785528 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.785646 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.785709 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.785872 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.785920 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260806 13:00:31.786974 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.781989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:33167 (local address 127.0.0.1:39132)
0806 13:00:31.782400 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:31.782409 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:31.782463 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:31.782710 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:31.782711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:31.782712 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:31.782754 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.782755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.783660 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.783661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.784994 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.784997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.786694 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.786695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.786766 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.786772 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 13:00:31.786838 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 13:00:31.786893 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":6,"thread_start_us":38,"threads_started":1}
I20260806 13:00:31.786892 11361 tablet_service.cc:1467] Processing CreateTablet for tablet 42b0e28f7e0c4091b4f0bb54974c7624 (DEFAULT_TABLE table=test-table [id=17874a88323d4de4aaca11acf46e96cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:31.787041 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.782195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39132 (local address 127.24.83.2:33167)
0806 13:00:31.782404 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.782408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.782452 (+ 44us) server_negotiation.cc:407] Connection header received
0806 13:00:31.782539 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.782541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.782556 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.782629 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:31.782834 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.783554 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.785745 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.786594 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.786679 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.786766 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.786852 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.786898 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
I20260806 13:00:31.787277 11361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42b0e28f7e0c4091b4f0bb54974c7624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:31.787294 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.781822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:43659 (local address 127.0.0.1:41744)
0806 13:00:31.782088 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:31.782126 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:31.782197 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:31.782503 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:31.782504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:31.782506 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:31.782561 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.782563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.785095 (+ 2532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.785095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.785779 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.785781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.787035 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.787036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.787116 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.787120 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:00:31.787179 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:00:31.787226 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":10,"thread_start_us":51,"threads_started":1}
I20260806 13:00:31.787351 11441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.781935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41744 (local address 127.24.83.1:43659)
0806 13:00:31.782104 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.782109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.782282 (+ 173us) server_negotiation.cc:407] Connection header received
0806 13:00:31.782328 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.782331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.782346 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.782420 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:31.782638 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.783746 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.786191 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.786884 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.786945 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.787000 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.787204 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.787255 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260806 13:00:31.787773 11222 tablet_service.cc:1467] Processing CreateTablet for tablet 42b0e28f7e0c4091b4f0bb54974c7624 (DEFAULT_TABLE table=test-table [id=17874a88323d4de4aaca11acf46e96cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:31.788125 11222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42b0e28f7e0c4091b4f0bb54974c7624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:31.788575 11084 tablet_service.cc:1467] Processing CreateTablet for tablet 42b0e28f7e0c4091b4f0bb54974c7624 (DEFAULT_TABLE table=test-table [id=17874a88323d4de4aaca11acf46e96cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:31.788880 11084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42b0e28f7e0c4091b4f0bb54974c7624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:31.790135 11444 tablet_bootstrap.cc:492] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0: Bootstrap starting.
I20260806 13:00:31.790758 11444 tablet_bootstrap.cc:654] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:31.791196 11444 log.cc:824] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:31.791486 11445 tablet_bootstrap.cc:492] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5: Bootstrap starting.
I20260806 13:00:31.792116 11445 tablet_bootstrap.cc:654] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:31.792132 11444 tablet_bootstrap.cc:492] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0: No bootstrap required, opened a new log
I20260806 13:00:31.792187 11444 ts_tablet_manager.cc:1397] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:31.792167 11447 tablet_bootstrap.cc:492] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b: Bootstrap starting.
I20260806 13:00:31.792665 11445 log.cc:824] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:31.792801 11447 tablet_bootstrap.cc:654] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:31.793330 11447 log.cc:824] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:31.794013 11447 tablet_bootstrap.cc:492] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b: No bootstrap required, opened a new log
I20260806 13:00:31.794092 11447 ts_tablet_manager.cc:1397] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:31.793991 11444 raft_consensus.cc:348] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41519 } } peers { permanent_uuid: "7ead6d42f73246d59fd03f1d9025945b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43659 } } peers { permanent_uuid: "aa103e644ada44c68bc53033bc1649e5" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33167 } }
I20260806 13:00:31.794179 11444 raft_consensus.cc:374] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:31.794224 11444 raft_consensus.cc:729] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d442f1b9cdc48af91d7ad172a4862f0, State: Initialized, Role: FOLLOWER
I20260806 13:00:31.794350 11444 consensus_queue.cc:260] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41519 } } peers { permanent_uuid: "7ead6d42f73246d59fd03f1d9025945b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43659 } } peers { permanent_uuid: "aa103e644ada44c68bc53033bc1649e5" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33167 } }
I20260806 13:00:31.794646 11430 heartbeater.cc:503] Master 127.24.83.62:36109 was elected leader, sending a full tablet report...
I20260806 13:00:31.794644 11444 ts_tablet_manager.cc:1428] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 13:00:31.795212 11445 tablet_bootstrap.cc:492] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5: No bootstrap required, opened a new log
I20260806 13:00:31.795267 11445 ts_tablet_manager.cc:1397] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 13:00:31.796042 11447 raft_consensus.cc:348] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41519 } } peers { permanent_uuid: "7ead6d42f73246d59fd03f1d9025945b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43659 } } peers { permanent_uuid: "aa103e644ada44c68bc53033bc1649e5" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33167 } }
I20260806 13:00:31.796224 11447 raft_consensus.cc:374] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:31.796262 11447 raft_consensus.cc:729] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ead6d42f73246d59fd03f1d9025945b, State: Initialized, Role: FOLLOWER
I20260806 13:00:31.796432 11447 consensus_queue.cc:260] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41519 } } peers { permanent_uuid: "7ead6d42f73246d59fd03f1d9025945b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43659 } } peers { permanent_uuid: "aa103e644ada44c68bc53033bc1649e5" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33167 } }
I20260806 13:00:31.796703 11447 ts_tablet_manager.cc:1428] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:00:31.796725 11452 raft_consensus.cc:482] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:00:31.796820 11152 heartbeater.cc:503] Master 127.24.83.62:36109 was elected leader, sending a full tablet report...
I20260806 13:00:31.796818 11452 raft_consensus.cc:504] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41519 } } peers { permanent_uuid: "7ead6d42f73246d59fd03f1d9025945b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43659 } } peers { permanent_uuid: "aa103e644ada44c68bc53033bc1649e5" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33167 } }
I20260806 13:00:31.797094 11452 leader_election.cc:288] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d442f1b9cdc48af91d7ad172a4862f0 (127.24.83.3:41519), aa103e644ada44c68bc53033bc1649e5 (127.24.83.2:33167)
I20260806 13:00:31.797247 11445 raft_consensus.cc:348] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41519 } } peers { permanent_uuid: "7ead6d42f73246d59fd03f1d9025945b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43659 } } peers { permanent_uuid: "aa103e644ada44c68bc53033bc1649e5" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33167 } }
I20260806 13:00:31.797399 11445 raft_consensus.cc:374] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:31.797422 11445 raft_consensus.cc:729] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa103e644ada44c68bc53033bc1649e5, State: Initialized, Role: FOLLOWER
I20260806 13:00:31.797539 11445 consensus_queue.cc:260] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41519 } } peers { permanent_uuid: "7ead6d42f73246d59fd03f1d9025945b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43659 } } peers { permanent_uuid: "aa103e644ada44c68bc53033bc1649e5" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33167 } }
I20260806 13:00:31.797806 11445 ts_tablet_manager.cc:1428] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:00:31.797806 11291 heartbeater.cc:503] Master 127.24.83.62:36109 was elected leader, sending a full tablet report...
I20260806 13:00:31.800586 11440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.797452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:60455 (local address 127.24.83.3:41519)
0806 13:00:31.797493 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.797495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.797502 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:00:31.797521 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.797522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.797532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.797581 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:31.797768 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.798462 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.799766 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.800283 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.800353 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.800362 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.800493 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.800527 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:00:31.800614 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.797247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:41519 (local address 127.24.83.1:60455)
0806 13:00:31.797306 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:31.797318 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:31.797356 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:31.797625 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:31.797626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:31.797628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:31.797693 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.797694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.798545 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.798546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.799683 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.799686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.800366 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.800366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.800423 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.800427 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:00:31.800470 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 13:00:31.800532 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 13:00:31.800892 11383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b0e28f7e0c4091b4f0bb54974c7624" candidate_uuid: "7ead6d42f73246d59fd03f1d9025945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" is_pre_election: true
I20260806 13:00:31.801020 11383 raft_consensus.cc:2454] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ead6d42f73246d59fd03f1d9025945b in term 0.
I20260806 13:00:31.801038 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.797367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:33167 (local address 127.24.83.1:58211)
0806 13:00:31.797539 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:31.797550 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:31.797604 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:31.798041 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:31.798042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:31.798044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:31.798103 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.798104 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.798938 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.798938 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.800031 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:31.800033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.800782 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.800783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:31.800856 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.800860 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:00:31.800929 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 13:00:31.800967 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260806 13:00:31.801060 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.797606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:58211 (local address 127.24.83.2:33167)
0806 13:00:31.797642 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.797644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.797651 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:00:31.797673 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.797674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.797683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.797747 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:31.798189 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.798845 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.800099 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.800669 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.800741 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.800751 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.800947 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.800991 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:00:31.801247 11037 leader_election.cc:302] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ead6d42f73246d59fd03f1d9025945b, 8d442f1b9cdc48af91d7ad172a4862f0; no voters:
I20260806 13:00:31.801371 11452 raft_consensus.cc:2781] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:00:31.801422 11452 raft_consensus.cc:482] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:00:31.801460 11452 raft_consensus.cc:3037] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:31.801432 11242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b0e28f7e0c4091b4f0bb54974c7624" candidate_uuid: "7ead6d42f73246d59fd03f1d9025945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa103e644ada44c68bc53033bc1649e5" is_pre_election: true
I20260806 13:00:31.801544 11242 raft_consensus.cc:2454] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ead6d42f73246d59fd03f1d9025945b in term 0.
I20260806 13:00:31.802088 11452 raft_consensus.cc:504] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41519 } } peers { permanent_uuid: "7ead6d42f73246d59fd03f1d9025945b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43659 } } peers { permanent_uuid: "aa103e644ada44c68bc53033bc1649e5" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33167 } }
I20260806 13:00:31.802213 11452 leader_election.cc:288] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [CANDIDATE]: Term 1 election: Requested vote from peers 8d442f1b9cdc48af91d7ad172a4862f0 (127.24.83.3:41519), aa103e644ada44c68bc53033bc1649e5 (127.24.83.2:33167)
I20260806 13:00:31.802338 11383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b0e28f7e0c4091b4f0bb54974c7624" candidate_uuid: "7ead6d42f73246d59fd03f1d9025945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d442f1b9cdc48af91d7ad172a4862f0"
I20260806 13:00:31.802363 11242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b0e28f7e0c4091b4f0bb54974c7624" candidate_uuid: "7ead6d42f73246d59fd03f1d9025945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa103e644ada44c68bc53033bc1649e5"
I20260806 13:00:31.802429 11383 raft_consensus.cc:3037] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:31.802435 11242 raft_consensus.cc:3037] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:31.803045 11383 raft_consensus.cc:2454] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ead6d42f73246d59fd03f1d9025945b in term 1.
I20260806 13:00:31.803102 11242 raft_consensus.cc:2454] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ead6d42f73246d59fd03f1d9025945b in term 1.
I20260806 13:00:31.803288 11037 leader_election.cc:302] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ead6d42f73246d59fd03f1d9025945b, aa103e644ada44c68bc53033bc1649e5; no voters:
I20260806 13:00:31.803380 11452 raft_consensus.cc:2781] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:31.803464 11452 raft_consensus.cc:686] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [term 1 LEADER]: Becoming Leader. State: Replica: 7ead6d42f73246d59fd03f1d9025945b, State: Running, Role: LEADER
I20260806 13:00:31.803542 11452 consensus_queue.cc:237] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41519 } } peers { permanent_uuid: "7ead6d42f73246d59fd03f1d9025945b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43659 } } peers { permanent_uuid: "aa103e644ada44c68bc53033bc1649e5" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33167 } }
I20260806 13:00:31.804270 10953 catalog_manager.cc:5495] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ead6d42f73246d59fd03f1d9025945b (127.24.83.1). New cstate: current_term: 1 leader_uuid: "7ead6d42f73246d59fd03f1d9025945b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ead6d42f73246d59fd03f1d9025945b" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa103e644ada44c68bc53033bc1649e5" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33167 } health_report { overall_health: UNKNOWN } } }
I20260806 13:00:31.809459 11441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.807386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41756 (local address 127.24.83.1:43659)
0806 13:00:31.807419 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.807421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.807444 (+ 23us) server_negotiation.cc:407] Connection header received
0806 13:00:31.807508 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.807509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.807523 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.807577 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:00:31.807661 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.808173 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.808862 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.809030 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.809078 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.809212 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:00:31.809282 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:00:31.809342 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.809391 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.809397 (+ 6us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:00:31.811039 11383 raft_consensus.cc:1261] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0 [term 1 FOLLOWER]: Refusing update from remote peer 7ead6d42f73246d59fd03f1d9025945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:31.811079 11242 raft_consensus.cc:1261] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5 [term 1 FOLLOWER]: Refusing update from remote peer 7ead6d42f73246d59fd03f1d9025945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:31.811411 11451 consensus_queue.cc:1035] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa103e644ada44c68bc53033bc1649e5" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 33167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:31.811546 11452 consensus_queue.cc:1035] T 42b0e28f7e0c4091b4f0bb54974c7624 P 7ead6d42f73246d59fd03f1d9025945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d442f1b9cdc48af91d7ad172a4862f0" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:31.813848 11460 mvcc.cc:205] Tried to move back new op lower bound from 7315542965495541760 to 7315542965467975680. Current Snapshot: MvccSnapshot[applied={T|T < 7315542965495541760}]
I20260806 13:00:31.813921 11459 mvcc.cc:205] Tried to move back new op lower bound from 7315542965495541760 to 7315542965467975680. Current Snapshot: MvccSnapshot[applied={T|T < 7315542965495541760}]
I20260806 13:00:31.815714 10952 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:31.815786 10952 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:31.816534 10952 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5ca26dcc1d5414d9776ded3ffe0b966: Sending DeleteTablet for 3 replicas of tablet 42b0e28f7e0c4091b4f0bb54974c7624
I20260806 13:00:31.816895 11222 tablet_service.cc:1514] Processing DeleteTablet for tablet 42b0e28f7e0c4091b4f0bb54974c7624 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:31 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:39132
I20260806 13:00:31.816897 11361 tablet_service.cc:1514] Processing DeleteTablet for tablet 42b0e28f7e0c4091b4f0bb54974c7624 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:31 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:45616
I20260806 13:00:31.817216 11084 tablet_service.cc:1514] Processing DeleteTablet for tablet 42b0e28f7e0c4091b4f0bb54974c7624 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:31 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:41744
I20260806 13:00:31.817277 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 11018
I20260806 13:00:31.819110 11468 tablet_replica.cc:330] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5: stopping tablet replica
I20260806 13:00:31.819235 11468 raft_consensus.cc:2229] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:00:31.819368 11468 raft_consensus.cc:2258] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:31.819653 11469 tablet_replica.cc:330] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0: stopping tablet replica
I20260806 13:00:31.819762 11469 raft_consensus.cc:2229] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:00:31.819852 11468 ts_tablet_manager.cc:1905] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:00:31.819911 11469 raft_consensus.cc:2258] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:31.820298 11469 ts_tablet_manager.cc:1905] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:00:31.821794 11469 ts_tablet_manager.cc:1918] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:00:31.821786 11468 ts_tablet_manager.cc:1918] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:00:31.821862 11468 log.cc:1196] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/42b0e28f7e0c4091b4f0bb54974c7624
I20260806 13:00:31.821856 11469 log.cc:1196] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/42b0e28f7e0c4091b4f0bb54974c7624
I20260806 13:00:31.822168 11469 ts_tablet_manager.cc:1939] T 42b0e28f7e0c4091b4f0bb54974c7624 P 8d442f1b9cdc48af91d7ad172a4862f0: Deleting consensus metadata
I20260806 13:00:31.822170 11468 ts_tablet_manager.cc:1939] T 42b0e28f7e0c4091b4f0bb54974c7624 P aa103e644ada44c68bc53033bc1649e5: Deleting consensus metadata
I20260806 13:00:31.822537 10938 catalog_manager.cc:4841] TS 8d442f1b9cdc48af91d7ad172a4862f0 (127.24.83.3:41519): tablet 42b0e28f7e0c4091b4f0bb54974c7624 (table test-table [id=17874a88323d4de4aaca11acf46e96cc]) successfully deleted
I20260806 13:00:31.822629 10938 catalog_manager.cc:4841] TS aa103e644ada44c68bc53033bc1649e5 (127.24.83.2:33167): tablet 42b0e28f7e0c4091b4f0bb54974c7624 (table test-table [id=17874a88323d4de4aaca11acf46e96cc]) successfully deleted
W20260806 13:00:31.825311 10937 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:43659 (error 108)
I20260806 13:00:31.825580 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 11157
I20260806 13:00:31.825769 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.825629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:43659 (local address 127.0.0.1:41768)
0806 13:00:31.825696 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:31.825733 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 13:00:31.826484 10937 catalog_manager.cc:4584] TS 7ead6d42f73246d59fd03f1d9025945b (127.24.83.1:43659): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 42b0e28f7e0c4091b4f0bb54974c7624: Network error: Client connection negotiation failed: client connection to 127.24.83.1:43659: connect: Connection refused (error 111)
I20260806 13:00:31.831106 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 11296
I20260806 13:00:31.837334 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 10920
2026-08-06T13:00:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1868 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T13:00:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:31Z Disabled control of system clock
I20260806 13:00:31.865279 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:43435
--webserver_interface=127.24.83.62
--webserver_port=0
--builtin_ntp_servers=127.24.83.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:43435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:00:31.945204 11475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:31.945412 11475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:31.945433 11475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:31.946501 11475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:31.946534 11475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:31.946547 11475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:31.946559 11475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:31.946571 11475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:31.947819 11475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:43435
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:43435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:31.948020 11475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:31.948246 11475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:31.952967 11482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:31.952996 11481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:31.953271 11475 server_base.cc:1034] running on GCE node
W20260806 13:00:31.952967 11484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:31.953585 11475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:31.953889 11475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:31.955052 11475 hybrid_clock.cc:648] HybridClock initialized: now 1786021231955041 us; error 42 us; skew 500 ppm
I20260806 13:00:31.956352 11475 webserver.cc:460] Webserver started at http://127.24.83.62:44635/ using document root <none> and password file <none>
I20260806 13:00:31.956604 11475 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:31.956674 11475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:31.956835 11475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:31.957815 11475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "b7892a629271483c878738320ae546b3"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "e5a9f201739c449bba7aaa708751a034"
server_key_iv: "8c41016c85a395c9239a04e44d330fe4"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.958174 11475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "b7892a629271483c878738320ae546b3"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "e5a9f201739c449bba7aaa708751a034"
server_key_iv: "8c41016c85a395c9239a04e44d330fe4"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.959507 11475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:00:31.960338 11490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:31.960533 11475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 13:00:31.960638 11475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "b7892a629271483c878738320ae546b3"
format_stamp: "Formatted at 2026-08-06 13:00:31 on dist-test-slave-zkpd"
server_key: "e5a9f201739c449bba7aaa708751a034"
server_key_iv: "8c41016c85a395c9239a04e44d330fe4"
server_key_version: "encryptionkey@0"
I20260806 13:00:31.960712 11475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:31.967258 11475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:31.967525 11475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:31.967674 11475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:31.971537 11475 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:43435
I20260806 13:00:31.971568 11542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:43435 every 8 connection(s)
I20260806 13:00:31.971877 11475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:31.972810 11543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:31.974936 11543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3: Bootstrap starting.
I20260806 13:00:31.975437 11543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:31.975677 11543 log.cc:824] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:31.976363 11543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3: No bootstrap required, opened a new log
I20260806 13:00:31.977867 11543 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7892a629271483c878738320ae546b3" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43435 } }
I20260806 13:00:31.977993 11543 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:31.978014 11543 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7892a629271483c878738320ae546b3, State: Initialized, Role: FOLLOWER
I20260806 13:00:31.978147 11543 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7892a629271483c878738320ae546b3" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43435 } }
I20260806 13:00:31.978227 11543 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:31.978250 11543 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:31.978277 11543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:31.978742 11543 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7892a629271483c878738320ae546b3" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43435 } }
I20260806 13:00:31.978825 11543 leader_election.cc:302] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7892a629271483c878738320ae546b3; no voters:
I20260806 13:00:31.978943 11543 leader_election.cc:288] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:31.979025 11546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:31.979233 11543 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:31.979287 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 11475
I20260806 13:00:31.979233 11546 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [term 1 LEADER]: Becoming Leader. State: Replica: b7892a629271483c878738320ae546b3, State: Running, Role: LEADER
I20260806 13:00:31.979405 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:31.979431 11546 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7892a629271483c878738320ae546b3" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43435 } }
I20260806 13:00:31.979610 24908 external_mini_cluster.cc:1428] Setting key cf83d82b59b66eb19050805aad7b8a1e
I20260806 13:00:31.979939 11548 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7892a629271483c878738320ae546b3. Latest consensus state: current_term: 1 leader_uuid: "b7892a629271483c878738320ae546b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7892a629271483c878738320ae546b3" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43435 } } }
I20260806 13:00:31.980036 11548 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:31.980013 11547 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7892a629271483c878738320ae546b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7892a629271483c878738320ae546b3" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 43435 } } }
I20260806 13:00:31.980249 11547 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:31.980724 11557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:31.981246 11557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:31.982788 11557 catalog_manager.cc:1347] Generated new cluster ID: bc6fc96c85ba4702ada14e617ec99d43
I20260806 13:00:31.982841 11557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:31.985602 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:31.980244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49464 (local address 127.24.83.62:43435)
0806 13:00:31.980634 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:31.980644 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:31.980681 (+ 37us) server_negotiation.cc:407] Connection header received
0806 13:00:31.980807 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:31.980814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:31.980891 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:31.981029 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:31.981278 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.982812 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.983672 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:31.984153 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:31.984292 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:31.984956 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:31.984960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:31.984960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:31.985130 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:31.985162 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:31.985172 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:31.985254 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:31.985340 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 13:00:31.985425 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":74,"threads_started":1}
I20260806 13:00:31.994660 11557 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:31.995258 11557 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:31.997512 11557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3: Generated new TSK 0
I20260806 13:00:31.997668 11557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:32.007031 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43435
--builtin_ntp_servers=127.24.83.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:00:32.090482 11569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:32.090662 11569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:32.090683 11569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:32.091794 11569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:32.091833 11569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:32.091858 11569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 13:00:32.093242 11569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:32.093499 11569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:32.093817 11569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:32.099642 11575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:32.099651 11574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:32.099833 11569 server_base.cc:1034] running on GCE node
W20260806 13:00:32.099658 11577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:32.100224 11569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:32.100508 11569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:32.101661 11569 hybrid_clock.cc:648] HybridClock initialized: now 1786021232101647 us; error 30 us; skew 500 ppm
I20260806 13:00:32.102984 11569 webserver.cc:460] Webserver started at http://127.24.83.1:38083/ using document root <none> and password file <none>
I20260806 13:00:32.103229 11569 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:32.103317 11569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:32.103468 11569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:32.104564 11569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "8ea96612d21a41b8a8b1a90fa55efc12"
format_stamp: "Formatted at 2026-08-06 13:00:32 on dist-test-slave-zkpd"
server_key: "fc303f0b218e8ffbf4c2ed194cda992f"
server_key_iv: "f85d14ec074fb5a4892d9986f09e6dce"
server_key_version: "encryptionkey@0"
I20260806 13:00:32.104914 11569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "8ea96612d21a41b8a8b1a90fa55efc12"
format_stamp: "Formatted at 2026-08-06 13:00:32 on dist-test-slave-zkpd"
server_key: "fc303f0b218e8ffbf4c2ed194cda992f"
server_key_iv: "f85d14ec074fb5a4892d9986f09e6dce"
server_key_version: "encryptionkey@0"
I20260806 13:00:32.106245 11569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:00:32.107158 11583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:32.107338 11569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:32.107445 11569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "8ea96612d21a41b8a8b1a90fa55efc12"
format_stamp: "Formatted at 2026-08-06 13:00:32 on dist-test-slave-zkpd"
server_key: "fc303f0b218e8ffbf4c2ed194cda992f"
server_key_iv: "f85d14ec074fb5a4892d9986f09e6dce"
server_key_version: "encryptionkey@0"
I20260806 13:00:32.107533 11569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:32.128166 11569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:32.128480 11569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:32.128636 11569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:32.128875 11569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:32.129211 11569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:32.129266 11569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:32.129305 11569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:32.129333 11569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:32.135221 11569 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:36007
I20260806 13:00:32.135277 11700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:36007 every 8 connection(s)
I20260806 13:00:32.135576 11569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 13:00:32.140661 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.136233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:45867 (local address 127.24.83.62:43435)
0806 13:00:32.136325 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:32.136327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:32.136528 (+ 201us) server_negotiation.cc:407] Connection header received
0806 13:00:32.137055 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:32.137056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:32.137071 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:32.137131 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:32.137799 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.138309 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.139288 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.139495 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.139573 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.140353 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:32.140354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:32.140355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:32.140360 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:32.140376 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:32.140380 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:32.140449 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:32.140546 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 13:00:32.140580 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 13:00:32.140877 11703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.136196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:43435 (local address 127.24.83.1:45867)
0806 13:00:32.136403 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:32.136436 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:32.136947 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:32.137204 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:32.137207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:32.137216 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:32.137701 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.137705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.138414 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.138415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.139189 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.139191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.139580 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.139581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.139709 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.140213 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:00:32.140213 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:32.140240 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:00:32.140261 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:32.140449 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:32.140456 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:32.140459 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:32.140546 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 13:00:32.140646 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260806 13:00:32.141440 11701 heartbeater.cc:347] Connected to a master server at 127.24.83.62:43435
I20260806 13:00:32.141516 11701 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:32.141703 11701 heartbeater.cc:511] Master 127.24.83.62:43435 requested a full tablet report, sending...
I20260806 13:00:32.142217 11507 ts_manager.cc:194] Registered new tserver with Master: 8ea96612d21a41b8a8b1a90fa55efc12 (127.24.83.1:36007)
I20260806 13:00:32.142838 11507 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.1:45867
I20260806 13:00:32.144946 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 11569
I20260806 13:00:32.145056 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 13:00:32.145203 24908 external_mini_cluster.cc:1428] Setting key d61a15210ba4a5d1dee8c73366f0b305
I20260806 13:00:32.145970 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43435
--builtin_ntp_servers=127.24.83.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:00:32.227463 11704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:32.227640 11704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:32.227660 11704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:32.228797 11704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:32.228834 11704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:32.228858 11704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 13:00:32.230043 11704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:32.230206 11704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:32.230430 11704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:32.235229 11710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:32.235215 11711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:32.235282 11704 server_base.cc:1034] running on GCE node
W20260806 13:00:32.235232 11713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:32.235620 11704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:32.235872 11704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:32.237079 11704 hybrid_clock.cc:648] HybridClock initialized: now 1786021232237068 us; error 26 us; skew 500 ppm
I20260806 13:00:32.238517 11704 webserver.cc:460] Webserver started at http://127.24.83.2:44533/ using document root <none> and password file <none>
I20260806 13:00:32.238778 11704 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:32.238827 11704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:32.239013 11704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:32.240116 11704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "b044f0c70faf44f49d735c1d385169c9"
format_stamp: "Formatted at 2026-08-06 13:00:32 on dist-test-slave-zkpd"
server_key: "cc50044867b5a767da52890cfa8e741d"
server_key_iv: "f6d746b02b5a13124414c82903e48feb"
server_key_version: "encryptionkey@0"
I20260806 13:00:32.240557 11704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "b044f0c70faf44f49d735c1d385169c9"
format_stamp: "Formatted at 2026-08-06 13:00:32 on dist-test-slave-zkpd"
server_key: "cc50044867b5a767da52890cfa8e741d"
server_key_iv: "f6d746b02b5a13124414c82903e48feb"
server_key_version: "encryptionkey@0"
I20260806 13:00:32.241904 11704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 13:00:32.242600 11719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:32.242817 11704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:32.242910 11704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "b044f0c70faf44f49d735c1d385169c9"
format_stamp: "Formatted at 2026-08-06 13:00:32 on dist-test-slave-zkpd"
server_key: "cc50044867b5a767da52890cfa8e741d"
server_key_iv: "f6d746b02b5a13124414c82903e48feb"
server_key_version: "encryptionkey@0"
I20260806 13:00:32.242993 11704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:32.277401 11704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:32.277725 11704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:32.277882 11704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:32.278126 11704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:32.278450 11704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:32.278506 11704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:32.278548 11704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:32.278575 11704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:32.284462 11704 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:45813
I20260806 13:00:32.284509 11837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:45813 every 8 connection(s)
I20260806 13:00:32.284837 11704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 13:00:32.289695 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.285572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:38795 (local address 127.24.83.62:43435)
0806 13:00:32.285676 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:32.285686 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:32.285892 (+ 206us) server_negotiation.cc:407] Connection header received
0806 13:00:32.286378 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:32.286379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:32.286390 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:32.286442 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:32.287127 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.287582 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.288479 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.288678 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.288732 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.289471 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:32.289472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:32.289473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:32.289477 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:32.289490 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:32.289494 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:32.289537 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:32.289615 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 13:00:32.289638 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 13:00:32.289944 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.285531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:43435 (local address 127.24.83.2:38795)
0806 13:00:32.285755 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:32.285787 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:32.286282 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:32.286502 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:32.286505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:32.286513 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:32.287052 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.287055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.287665 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.287666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.288408 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.288410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.288755 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.288755 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.288856 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.289349 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:00:32.289349 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:32.289375 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:00:32.289396 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:32.289548 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:32.289552 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:32.289555 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:32.289604 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 13:00:32.289678 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":124,"threads_started":1}
I20260806 13:00:32.290484 11838 heartbeater.cc:347] Connected to a master server at 127.24.83.62:43435
I20260806 13:00:32.290608 11838 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:32.290818 11838 heartbeater.cc:511] Master 127.24.83.62:43435 requested a full tablet report, sending...
I20260806 13:00:32.290980 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 11704
I20260806 13:00:32.291090 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 13:00:32.291244 11507 ts_manager.cc:194] Registered new tserver with Master: b044f0c70faf44f49d735c1d385169c9 (127.24.83.2:45813)
I20260806 13:00:32.291297 24908 external_mini_cluster.cc:1428] Setting key e67a2e624d9f8d4df078a326d0a45e37
I20260806 13:00:32.291703 11507 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.2:38795
I20260806 13:00:32.292434 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43435
--builtin_ntp_servers=127.24.83.20:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:00:32.373255 11841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:32.373456 11841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:32.373476 11841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:32.374634 11841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:32.374675 11841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:32.374732 11841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 13:00:32.376106 11841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:43435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:32.376343 11841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:32.376636 11841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:32.381537 11846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:32.381531 11850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:32.381560 11847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:32.381775 11841 server_base.cc:1034] running on GCE node
I20260806 13:00:32.382035 11841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:32.382280 11841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:32.383435 11841 hybrid_clock.cc:648] HybridClock initialized: now 1786021232383425 us; error 40 us; skew 500 ppm
I20260806 13:00:32.388942 11841 webserver.cc:460] Webserver started at http://127.24.83.3:37529/ using document root <none> and password file <none>
I20260806 13:00:32.389185 11841 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:32.389271 11841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:32.389426 11841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:32.390487 11841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "a8d2e27957c24febb66c0c82b21e6c7d"
format_stamp: "Formatted at 2026-08-06 13:00:32 on dist-test-slave-zkpd"
server_key: "fac22a8fe108353052b5ee3ebfaf33f2"
server_key_iv: "1ddfa4f379ac509be2d1403c0ab0dbce"
server_key_version: "encryptionkey@0"
I20260806 13:00:32.390847 11841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "a8d2e27957c24febb66c0c82b21e6c7d"
format_stamp: "Formatted at 2026-08-06 13:00:32 on dist-test-slave-zkpd"
server_key: "fac22a8fe108353052b5ee3ebfaf33f2"
server_key_iv: "1ddfa4f379ac509be2d1403c0ab0dbce"
server_key_version: "encryptionkey@0"
I20260806 13:00:32.392189 11841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:00:32.392897 11859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:32.393074 11841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:32.393177 11841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "a8d2e27957c24febb66c0c82b21e6c7d"
format_stamp: "Formatted at 2026-08-06 13:00:32 on dist-test-slave-zkpd"
server_key: "fac22a8fe108353052b5ee3ebfaf33f2"
server_key_iv: "1ddfa4f379ac509be2d1403c0ab0dbce"
server_key_version: "encryptionkey@0"
I20260806 13:00:32.393256 11841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:32.411027 11841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:32.411320 11841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:32.411469 11841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:32.411692 11841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:32.412081 11841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:32.412137 11841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:32.412170 11841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:32.412197 11841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:32.417927 11841 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:41779
I20260806 13:00:32.417976 11975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:41779 every 8 connection(s)
I20260806 13:00:32.418292 11841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 13:00:32.423421 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.418974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:44997 (local address 127.24.83.62:43435)
0806 13:00:32.419020 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:32.419022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:32.419284 (+ 262us) server_negotiation.cc:407] Connection header received
0806 13:00:32.419818 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:32.419819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:32.419832 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:32.419901 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:32.420581 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.421086 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.422054 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.422259 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.422334 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.423101 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:32.423103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:32.423103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:32.423108 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:32.423122 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:32.423127 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:32.423195 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:32.423304 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 13:00:32.423335 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:00:32.423615 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.418965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:43435 (local address 127.24.83.3:44997)
0806 13:00:32.419162 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:32.419195 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:32.419707 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:32.419976 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:32.419980 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:32.419997 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:32.420481 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.420486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.421185 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.421186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.421955 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.421957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.422354 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.422355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.422488 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.422960 (+ 472us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:00:32.422961 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:32.422989 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:00:32.423010 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:32.423197 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:32.423204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:32.423207 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:32.423291 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 13:00:32.423391 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260806 13:00:32.424157 11976 heartbeater.cc:347] Connected to a master server at 127.24.83.62:43435
I20260806 13:00:32.424252 11976 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:32.424427 11976 heartbeater.cc:511] Master 127.24.83.62:43435 requested a full tablet report, sending...
I20260806 13:00:32.424815 11506 ts_manager.cc:194] Registered new tserver with Master: a8d2e27957c24febb66c0c82b21e6c7d (127.24.83.3:41779)
I20260806 13:00:32.425138 11506 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.3:44997
I20260806 13:00:32.427541 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 11841
I20260806 13:00:32.427670 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 13:00:32.427860 24908 external_mini_cluster.cc:1428] Setting key d0e800a5cb221f1a789fc414958519d8
I20260806 13:00:32.428714 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:00:32.432647 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.429976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.24.83.62:43435)
0806 13:00:32.430029 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:32.430031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:32.430116 (+ 85us) server_negotiation.cc:407] Connection header received
0806 13:00:32.430156 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:32.430156 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:32.430165 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:32.430206 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:32.430333 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.430770 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.431461 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.431629 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.431703 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.432360 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:32.432361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:32.432361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:32.432366 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:32.432378 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:32.432382 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:32.432492 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:32.432560 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 13:00:32.432570 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 13:00:32.434348 11506 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:00:32.435768 11506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:00:32.441944 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.437853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:41779 (local address 127.0.0.1:40306)
0806 13:00:32.438411 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:32.438419 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:32.438493 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:32.438743 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:32.438744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:32.438747 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:32.438790 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.438795 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.440015 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.440016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.440901 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.440904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.441654 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.441655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.441735 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.441748 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 13:00:32.441825 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 13:00:32.441872 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":31,"thread_start_us":103,"threads_started":1}
I20260806 13:00:32.441977 11987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.437995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.24.83.3:41779)
0806 13:00:32.438358 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:32.438361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:32.438487 (+ 126us) server_negotiation.cc:407] Connection header received
0806 13:00:32.438566 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:32.438568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:32.438590 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:32.438682 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:32.438871 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.439931 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.440988 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.441572 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.441643 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.441721 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:32.441852 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 13:00:32.441897 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":75,"threads_started":1}
I20260806 13:00:32.442523 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.437904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:36007 (local address 127.0.0.1:46252)
0806 13:00:32.438579 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:32.438588 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:32.438635 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:32.438986 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:32.438986 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:32.438988 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:32.439030 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.439032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.439918 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.439919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.441205 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.441208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.442271 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.442272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.442331 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.442334 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:32.442417 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 13:00:32.442457 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260806 13:00:32.442560 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.438114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.24.83.1:36007)
0806 13:00:32.438299 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:32.438304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:32.438652 (+ 348us) server_negotiation.cc:407] Connection header received
0806 13:00:32.438822 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:32.438824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:32.438840 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:32.438912 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:32.439105 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.439807 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.441295 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.442064 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.442135 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.442237 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:32.442436 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 13:00:32.442477 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260806 13:00:32.442636 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.437850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:45813 (local address 127.0.0.1:53028)
0806 13:00:32.438021 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:32.438062 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:32.438127 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:32.438514 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:32.438517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:32.438519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:32.438587 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.438589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.439977 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.439977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.441205 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.441208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.442436 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.442437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.442498 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.442503 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:00:32.442532 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 13:00:32.442571 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":66,"thread_start_us":63,"threads_started":1}
I20260806 13:00:32.442807 11908 tablet_service.cc:1467] Processing CreateTablet for tablet 7833793aeee343ef95e35b08cc7807c7 (DEFAULT_TABLE table=test-table [id=c6903cd07cf6467684f7beb6f0fa9775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:32.443073 11989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.438155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53028 (local address 127.24.83.2:45813)
0806 13:00:32.438346 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:32.438349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:32.438358 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:00:32.438393 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:32.438395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:32.438407 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:32.438493 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:32.438692 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.439862 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.441381 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.442334 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.442829 (+ 495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.442922 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:32.442950 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 13:00:32.442991 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
I20260806 13:00:32.443169 11908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7833793aeee343ef95e35b08cc7807c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:32.443346 11632 tablet_service.cc:1467] Processing CreateTablet for tablet 7833793aeee343ef95e35b08cc7807c7 (DEFAULT_TABLE table=test-table [id=c6903cd07cf6467684f7beb6f0fa9775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:32.443681 11632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7833793aeee343ef95e35b08cc7807c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:32.443686 11768 tablet_service.cc:1467] Processing CreateTablet for tablet 7833793aeee343ef95e35b08cc7807c7 (DEFAULT_TABLE table=test-table [id=c6903cd07cf6467684f7beb6f0fa9775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:32.444023 11768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7833793aeee343ef95e35b08cc7807c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:32.445818 11990 tablet_bootstrap.cc:492] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: Bootstrap starting.
I20260806 13:00:32.446650 11990 tablet_bootstrap.cc:654] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:32.447113 11990 log.cc:824] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:32.447472 11992 tablet_bootstrap.cc:492] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: Bootstrap starting.
I20260806 13:00:32.447816 11990 tablet_bootstrap.cc:492] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: No bootstrap required, opened a new log
I20260806 13:00:32.447872 11990 ts_tablet_manager.cc:1397] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:32.448091 11992 tablet_bootstrap.cc:654] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:32.448375 11992 log.cc:824] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:32.448731 11991 tablet_bootstrap.cc:492] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12: Bootstrap starting.
I20260806 13:00:32.449069 11992 tablet_bootstrap.cc:492] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: No bootstrap required, opened a new log
I20260806 13:00:32.449121 11992 ts_tablet_manager.cc:1397] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:32.449841 11990 raft_consensus.cc:348] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b044f0c70faf44f49d735c1d385169c9" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45813 } } peers { permanent_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41779 } } peers { permanent_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 36007 } }
I20260806 13:00:32.450088 11990 raft_consensus.cc:374] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:32.450145 11990 raft_consensus.cc:729] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8d2e27957c24febb66c0c82b21e6c7d, State: Initialized, Role: FOLLOWER
I20260806 13:00:32.450268 11990 consensus_queue.cc:260] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b044f0c70faf44f49d735c1d385169c9" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45813 } } peers { permanent_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41779 } } peers { permanent_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 36007 } }
I20260806 13:00:32.450543 11990 ts_tablet_manager.cc:1428] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 13:00:32.450565 11976 heartbeater.cc:503] Master 127.24.83.62:43435 was elected leader, sending a full tablet report...
I20260806 13:00:32.451395 11992 raft_consensus.cc:348] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b044f0c70faf44f49d735c1d385169c9" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45813 } } peers { permanent_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41779 } } peers { permanent_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 36007 } }
I20260806 13:00:32.451550 11992 raft_consensus.cc:374] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:32.451586 11992 raft_consensus.cc:729] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b044f0c70faf44f49d735c1d385169c9, State: Initialized, Role: FOLLOWER
I20260806 13:00:32.451706 11992 consensus_queue.cc:260] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b044f0c70faf44f49d735c1d385169c9" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45813 } } peers { permanent_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41779 } } peers { permanent_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 36007 } }
I20260806 13:00:32.452024 11992 ts_tablet_manager.cc:1428] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 13:00:32.452056 11838 heartbeater.cc:503] Master 127.24.83.62:43435 was elected leader, sending a full tablet report...
I20260806 13:00:32.452709 11991 tablet_bootstrap.cc:654] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:32.453058 11991 log.cc:824] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:32.453720 11991 tablet_bootstrap.cc:492] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12: No bootstrap required, opened a new log
I20260806 13:00:32.453773 11991 ts_tablet_manager.cc:1397] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 13:00:32.455632 11991 raft_consensus.cc:348] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b044f0c70faf44f49d735c1d385169c9" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45813 } } peers { permanent_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41779 } } peers { permanent_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 36007 } }
I20260806 13:00:32.455746 11991 raft_consensus.cc:374] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:32.455770 11991 raft_consensus.cc:729] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea96612d21a41b8a8b1a90fa55efc12, State: Initialized, Role: FOLLOWER
I20260806 13:00:32.455935 11991 consensus_queue.cc:260] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b044f0c70faf44f49d735c1d385169c9" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45813 } } peers { permanent_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41779 } } peers { permanent_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 36007 } }
I20260806 13:00:32.456198 11991 ts_tablet_manager.cc:1428] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:00:32.456223 11701 heartbeater.cc:503] Master 127.24.83.62:43435 was elected leader, sending a full tablet report...
W20260806 13:00:32.535375 11839 tablet.cc:2369] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:00:32.636363 11702 tablet.cc:2369] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:00:32.668836 11977 tablet.cc:2369] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:00:32.706718 11999 raft_consensus.cc:482] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:00:32.706832 11999 raft_consensus.cc:504] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b044f0c70faf44f49d735c1d385169c9" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45813 } } peers { permanent_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41779 } } peers { permanent_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 36007 } }
I20260806 13:00:32.707231 11999 leader_election.cc:288] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b044f0c70faf44f49d735c1d385169c9 (127.24.83.2:45813), a8d2e27957c24febb66c0c82b21e6c7d (127.24.83.3:41779)
I20260806 13:00:32.711438 11989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.707502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:59013 (local address 127.24.83.2:45813)
0806 13:00:32.707551 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:32.707554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:32.707615 (+ 61us) server_negotiation.cc:407] Connection header received
0806 13:00:32.707770 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:32.707771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:32.707786 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:32.707854 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:32.708088 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.708875 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.710466 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.711041 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.711128 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.711137 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:32.711326 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 13:00:32.711367 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:00:32.711438 11987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.707502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:48079 (local address 127.24.83.3:41779)
0806 13:00:32.707603 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:32.707605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:32.707762 (+ 157us) server_negotiation.cc:407] Connection header received
0806 13:00:32.707822 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:32.707824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:32.707834 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:32.707905 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:32.708081 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.708875 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.710466 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.711044 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.711128 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.711137 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:32.711326 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 13:00:32.711367 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:00:32.711465 12001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.707433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:41779 (local address 127.24.83.1:48079)
0806 13:00:32.707667 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:32.707677 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:32.707738 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:32.707925 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:32.707926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:32.707927 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:32.707996 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.707997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.708961 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.708962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.710371 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.710376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.711141 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.711142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.711234 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.711240 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 13:00:32.711304 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 13:00:32.711355 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":57,"thread_start_us":57,"threads_started":1}
I20260806 13:00:32.711465 12000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.707430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:45813 (local address 127.24.83.1:59013)
0806 13:00:32.707552 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:32.707563 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:32.707646 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:32.707869 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:32.707870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:32.707874 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:32.707973 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.707976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.708961 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.708962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.710371 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:32.710376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.711139 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.711141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:32.711234 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.711242 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 13:00:32.711301 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:00:32.711355 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":103,"thread_start_us":58,"threads_started":1}
I20260806 13:00:32.711846 11928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7833793aeee343ef95e35b08cc7807c7" candidate_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" is_pre_election: true
I20260806 13:00:32.712023 11928 raft_consensus.cc:2454] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ea96612d21a41b8a8b1a90fa55efc12 in term 0.
I20260806 13:00:32.711977 11791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7833793aeee343ef95e35b08cc7807c7" candidate_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b044f0c70faf44f49d735c1d385169c9" is_pre_election: true
I20260806 13:00:32.712167 11791 raft_consensus.cc:2454] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ea96612d21a41b8a8b1a90fa55efc12 in term 0.
I20260806 13:00:32.712306 11587 leader_election.cc:302] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea96612d21a41b8a8b1a90fa55efc12, a8d2e27957c24febb66c0c82b21e6c7d; no voters:
I20260806 13:00:32.712522 11999 raft_consensus.cc:2781] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:00:32.712571 11999 raft_consensus.cc:482] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:00:32.712586 11999 raft_consensus.cc:3037] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:32.713326 11999 raft_consensus.cc:504] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b044f0c70faf44f49d735c1d385169c9" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45813 } } peers { permanent_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41779 } } peers { permanent_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 36007 } }
I20260806 13:00:32.713472 11999 leader_election.cc:288] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [CANDIDATE]: Term 1 election: Requested vote from peers b044f0c70faf44f49d735c1d385169c9 (127.24.83.2:45813), a8d2e27957c24febb66c0c82b21e6c7d (127.24.83.3:41779)
I20260806 13:00:32.713649 11928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7833793aeee343ef95e35b08cc7807c7" candidate_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8d2e27957c24febb66c0c82b21e6c7d"
I20260806 13:00:32.713725 11928 raft_consensus.cc:3037] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:32.714447 11928 raft_consensus.cc:2454] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea96612d21a41b8a8b1a90fa55efc12 in term 1.
I20260806 13:00:32.714567 11791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7833793aeee343ef95e35b08cc7807c7" candidate_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b044f0c70faf44f49d735c1d385169c9"
I20260806 13:00:32.714600 11587 leader_election.cc:302] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea96612d21a41b8a8b1a90fa55efc12, a8d2e27957c24febb66c0c82b21e6c7d; no voters:
I20260806 13:00:32.714663 11791 raft_consensus.cc:3037] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:32.714749 11999 raft_consensus.cc:2781] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:32.714928 11999 raft_consensus.cc:686] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [term 1 LEADER]: Becoming Leader. State: Replica: 8ea96612d21a41b8a8b1a90fa55efc12, State: Running, Role: LEADER
I20260806 13:00:32.715032 11999 consensus_queue.cc:237] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b044f0c70faf44f49d735c1d385169c9" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45813 } } peers { permanent_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41779 } } peers { permanent_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 36007 } }
I20260806 13:00:32.715652 11791 raft_consensus.cc:2454] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea96612d21a41b8a8b1a90fa55efc12 in term 1.
I20260806 13:00:32.715957 11506 catalog_manager.cc:5495] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ea96612d21a41b8a8b1a90fa55efc12 (127.24.83.1). New cstate: current_term: 1 leader_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b044f0c70faf44f49d735c1d385169c9" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea96612d21a41b8a8b1a90fa55efc12" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 36007 } health_report { overall_health: HEALTHY } } }
I20260806 13:00:32.722968 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.720903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46264 (local address 127.24.83.1:36007)
0806 13:00:32.720956 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:32.720958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:32.720965 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:00:32.720992 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:32.720993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:32.721009 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:32.721061 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:00:32.721178 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.721613 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.722285 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:32.722456 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:32.722503 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:32.722652 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:00:32.722732 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:00:32.722798 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:32.722851 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 13:00:32.722860 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 13:00:32.724606 11791 raft_consensus.cc:1261] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9 [term 1 FOLLOWER]: Refusing update from remote peer 8ea96612d21a41b8a8b1a90fa55efc12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:32.724622 11928 raft_consensus.cc:1261] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d [term 1 FOLLOWER]: Refusing update from remote peer 8ea96612d21a41b8a8b1a90fa55efc12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:32.724920 11999 consensus_queue.cc:1035] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8d2e27957c24febb66c0c82b21e6c7d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 41779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:32.725121 12002 consensus_queue.cc:1035] T 7833793aeee343ef95e35b08cc7807c7 P 8ea96612d21a41b8a8b1a90fa55efc12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b044f0c70faf44f49d735c1d385169c9" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 45813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:32.727661 12005 mvcc.cc:205] Tried to move back new op lower bound from 7315542969237819392 to 7315542969201430528. Current Snapshot: MvccSnapshot[applied={T|T < 7315542969237819392}]
I20260806 13:00:32.728137 12006 mvcc.cc:205] Tried to move back new op lower bound from 7315542969237819392 to 7315542969201430528. Current Snapshot: MvccSnapshot[applied={T|T < 7315542969237819392}]
I20260806 13:00:32.728317 12007 mvcc.cc:205] Tried to move back new op lower bound from 7315542969237819392 to 7315542969201430528. Current Snapshot: MvccSnapshot[applied={T|T < 7315542969237819392 or (T in {7315542969237819392})}]
I20260806 13:00:32.730582 11506 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:32.730679 11506 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:32.731431 11506 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7892a629271483c878738320ae546b3: Sending DeleteTablet for 3 replicas of tablet 7833793aeee343ef95e35b08cc7807c7
I20260806 13:00:32.731763 11908 tablet_service.cc:1514] Processing DeleteTablet for tablet 7833793aeee343ef95e35b08cc7807c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:32 UTC) from {username='slave'} at 127.0.0.1:40306
I20260806 13:00:32.731781 11768 tablet_service.cc:1514] Processing DeleteTablet for tablet 7833793aeee343ef95e35b08cc7807c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:32 UTC) from {username='slave'} at 127.0.0.1:53028
I20260806 13:00:32.732004 11632 tablet_service.cc:1514] Processing DeleteTablet for tablet 7833793aeee343ef95e35b08cc7807c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:32 UTC) from {username='slave'} at 127.0.0.1:46252
I20260806 13:00:32.732118 12015 tablet_replica.cc:330] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: stopping tablet replica
I20260806 13:00:32.732185 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 11569
I20260806 13:00:32.732226 12015 raft_consensus.cc:2229] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:00:32.733534 12016 tablet_replica.cc:330] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: stopping tablet replica
I20260806 13:00:32.733659 12016 raft_consensus.cc:2229] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:00:32.733793 12016 raft_consensus.cc:2258] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:32.734244 12016 ts_tablet_manager.cc:1905] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:00:32.735987 12016 ts_tablet_manager.cc:1918] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:00:32.736059 12016 log.cc:1196] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/7833793aeee343ef95e35b08cc7807c7
I20260806 13:00:32.736284 12016 ts_tablet_manager.cc:1939] T 7833793aeee343ef95e35b08cc7807c7 P b044f0c70faf44f49d735c1d385169c9: Deleting consensus metadata
I20260806 13:00:32.739234 11493 catalog_manager.cc:4841] TS b044f0c70faf44f49d735c1d385169c9 (127.24.83.2:45813): tablet 7833793aeee343ef95e35b08cc7807c7 (table test-table [id=c6903cd07cf6467684f7beb6f0fa9775]) successfully deleted
I20260806 13:00:32.739758 12015 raft_consensus.cc:2258] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 13:00:32.739938 11492 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:36007 (error 108)
I20260806 13:00:32.740281 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 11704
I20260806 13:00:32.740296 12015 ts_tablet_manager.cc:1905] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:00:32.740392 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:32.740262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:36007 (local address 127.0.0.1:46266)
0806 13:00:32.740311 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:32.740351 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:36007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 13:00:32.741284 11492 catalog_manager.cc:4584] TS 8ea96612d21a41b8a8b1a90fa55efc12 (127.24.83.1:36007): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7833793aeee343ef95e35b08cc7807c7: Network error: Client connection negotiation failed: client connection to 127.24.83.1:36007: connect: Connection refused (error 111)
I20260806 13:00:32.742027 12015 ts_tablet_manager.cc:1918] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:00:32.742097 12015 log.cc:1196] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/7833793aeee343ef95e35b08cc7807c7
I20260806 13:00:32.742336 12015 ts_tablet_manager.cc:1939] T 7833793aeee343ef95e35b08cc7807c7 P a8d2e27957c24febb66c0c82b21e6c7d: Deleting consensus metadata
I20260806 13:00:32.742763 11494 catalog_manager.cc:4841] TS a8d2e27957c24febb66c0c82b21e6c7d (127.24.83.3:41779): tablet 7833793aeee343ef95e35b08cc7807c7 (table test-table [id=c6903cd07cf6467684f7beb6f0fa9775]) successfully deleted
I20260806 13:00:32.745893 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 11841
I20260806 13:00:32.750919 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 11475
2026-08-06T13:00:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (912 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:32 dist-test-slave-zkpd krb5kdc[12021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:32 dist-test-slave-zkpd krb5kdc[12021](info): set up 2 sockets
Aug 06 13:00:32 dist-test-slave-zkpd krb5kdc[12021](info): commencing operation
krb5kdc: starting...
W20260806 13:00:33.796222 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:33 dist-test-slave-zkpd krb5kdc[12021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 13:00:33.910671 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45117
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:45627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:45117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:33.990417 12034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:33.990608 12034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:33.990629 12034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:33.991765 12034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:33.991808 12034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:33.991822 12034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:33.991834 12034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:33.991884 12034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:33.993335 12034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:45627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:45117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:33.993568 12034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:33.993818 12034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:33.998659 12039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:33.998689 12042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:33.998626 12040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:33.998833 12034 server_base.cc:1034] running on GCE node
I20260806 13:00:33.999069 12034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:33.999388 12034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:34.000557 12034 hybrid_clock.cc:648] HybridClock initialized: now 1786021234000543 us; error 26 us; skew 500 ppm
Aug 06 13:00:33 dist-test-slave-zkpd krb5kdc[12021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021233, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:34.002908 12034 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:00:34.003274 12034 webserver.cc:460] Webserver started at http://127.24.83.62:35735/ using document root <none> and password file <none>
I20260806 13:00:34.003516 12034 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:34.003562 12034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:34.003712 12034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:34.004802 12034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "05be91c70a4b4d5c946e1cf75ae777cf"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "cc31286f6d959e170ab596150ff18fc4"
server_key_iv: "61ee9b103e88cdf7bdffd0c2173c0b2b"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.005157 12034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "05be91c70a4b4d5c946e1cf75ae777cf"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "cc31286f6d959e170ab596150ff18fc4"
server_key_iv: "61ee9b103e88cdf7bdffd0c2173c0b2b"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.006451 12034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:00:34.007153 12049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:34.007336 12034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:34.007444 12034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "05be91c70a4b4d5c946e1cf75ae777cf"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "cc31286f6d959e170ab596150ff18fc4"
server_key_iv: "61ee9b103e88cdf7bdffd0c2173c0b2b"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.007519 12034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:34.025399 12034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:34.027321 12034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:34.027581 12034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:34.031530 12034 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:45117
I20260806 13:00:34.031565 12102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:45117 every 8 connection(s)
I20260806 13:00:34.031968 12034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:34.032845 12103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:34.035063 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 12034
I20260806 13:00:34.035084 12103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf: Bootstrap starting.
I20260806 13:00:34.035156 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:34.035372 24908 external_mini_cluster.cc:1428] Setting key e61b024547bfb43d209fbc3f25dba5ee
I20260806 13:00:34.035733 12103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:34.036106 12103 log.cc:824] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:34.036815 12103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf: No bootstrap required, opened a new log
I20260806 13:00:34.038558 12103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05be91c70a4b4d5c946e1cf75ae777cf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45117 } }
I20260806 13:00:34.038717 12103 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:34.038784 12103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05be91c70a4b4d5c946e1cf75ae777cf, State: Initialized, Role: FOLLOWER
I20260806 13:00:34.038942 12103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05be91c70a4b4d5c946e1cf75ae777cf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45117 } }
I20260806 13:00:34.039040 12103 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:34.039083 12103 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:34.039134 12103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:34.039642 12103 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05be91c70a4b4d5c946e1cf75ae777cf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45117 } }
I20260806 13:00:34.039772 12103 leader_election.cc:302] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05be91c70a4b4d5c946e1cf75ae777cf; no voters:
I20260806 13:00:34.039983 12103 leader_election.cc:288] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:34.040027 12109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:34.040253 12109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [term 1 LEADER]: Becoming Leader. State: Replica: 05be91c70a4b4d5c946e1cf75ae777cf, State: Running, Role: LEADER
I20260806 13:00:34.040333 12103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:34.040395 12109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05be91c70a4b4d5c946e1cf75ae777cf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45117 } }
I20260806 13:00:34.040690 12109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05be91c70a4b4d5c946e1cf75ae777cf. Latest consensus state: current_term: 1 leader_uuid: "05be91c70a4b4d5c946e1cf75ae777cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05be91c70a4b4d5c946e1cf75ae777cf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45117 } } }
I20260806 13:00:34.040743 12110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05be91c70a4b4d5c946e1cf75ae777cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05be91c70a4b4d5c946e1cf75ae777cf" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45117 } } }
I20260806 13:00:34.040830 12109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:34.040863 12110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:34.041143 12117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:34.041641 12117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 13:00:34 dist-test-slave-zkpd krb5kdc[12021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:34.043730 12117 catalog_manager.cc:1347] Generated new cluster ID: 41368753840f4254a0db4f2641cdebf1
I20260806 13:00:34.043785 12117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:34.046522 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.035909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.24.83.62:45117)
0806 13:00:34.036109 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.036118 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.036169 (+ 51us) server_negotiation.cc:407] Connection header received
0806 13:00:34.036734 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.036742 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.036790 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.036955 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:34.037588 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.038452 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.039447 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.039953 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.040081 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.043104 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:34.043108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:34.043108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:34.043120 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:34.045002 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:34.045385 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:34.045389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:34.045389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:34.045482 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:34.045695 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:34.045696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:34.045696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:34.045925 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:34.046103 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.046179 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.046311 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
I20260806 13:00:34.052613 12117 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:34.053085 12117 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:34.058252 12117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf: Generated new TSK 0
I20260806 13:00:34.058432 12117 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.83.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
Entry for principal kudu/127.24.83.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab.
I20260806 13:00:34.092374 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45117
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:45627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:34.183917 12131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:34.184111 12131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:34.184136 12131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:34.185284 12131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:34.185326 12131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:34.185352 12131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 13:00:34.186789 12131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:45627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.1.keytab
--principal=kudu/127.24.83.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:34.187021 12131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:34.187242 12131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:34.192164 12138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:34.192281 12131 server_base.cc:1034] running on GCE node
W20260806 13:00:34.192121 12140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:34.192126 12137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:34.192587 12131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:34.192868 12131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:34.194010 12131 hybrid_clock.cc:648] HybridClock initialized: now 1786021234193993 us; error 36 us; skew 500 ppm
Aug 06 13:00:34 dist-test-slave-zkpd krb5kdc[12021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021234, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:34.196419 12131 init.cc:377] Logged in from keytab as kudu/127.24.83.1@KRBTEST.COM (short username kudu)
I20260806 13:00:34.196888 12131 webserver.cc:460] Webserver started at http://127.24.83.1:38253/ using document root <none> and password file <none>
I20260806 13:00:34.197110 12131 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:34.197154 12131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:34.197294 12131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:34.198329 12131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "fcd8076128424c66a2732d79058f4112"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "53e70245f67648eff1a966ae30fbccfc"
server_key_iv: "8e39c208e1f775d4adbd20eb6ae13b1e"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.198688 12131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "fcd8076128424c66a2732d79058f4112"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "53e70245f67648eff1a966ae30fbccfc"
server_key_iv: "8e39c208e1f775d4adbd20eb6ae13b1e"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.200392 12131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:00:34.201294 12149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:34.201542 12131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:00:34.201691 12131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "fcd8076128424c66a2732d79058f4112"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "53e70245f67648eff1a966ae30fbccfc"
server_key_iv: "8e39c208e1f775d4adbd20eb6ae13b1e"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.201787 12131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:34.209297 12131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:34.211236 12131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:34.211396 12131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:34.211635 12131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:34.211999 12131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:34.212036 12131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:34.212074 12131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:34.212100 12131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:34.218204 12131 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:44729
I20260806 13:00:34.218264 12270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:44729 every 8 connection(s)
I20260806 13:00:34.218557 12131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
Aug 06 13:00:34 dist-test-slave-zkpd krb5kdc[12021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021234, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.1@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:34.227504 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.219222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:56903 (local address 127.24.83.62:45117)
0806 13:00:34.219290 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.219292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.219509 (+ 217us) server_negotiation.cc:407] Connection header received
0806 13:00:34.220092 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.220093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.220105 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.220169 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:34.221096 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.221626 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.222622 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.222852 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.222935 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.224948 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:34.224950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:34.224950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:34.224958 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:34.226132 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:34.226527 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:34.226529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:34.226529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:34.226592 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:34.226856 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:34.226857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:34.226858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:34.226979 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:34.227056 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.227321 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.227412 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 13:00:34.227676 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.219216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45117 (local address 127.24.83.1:56903)
0806 13:00:34.219370 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:34.219404 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:34.219986 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:34.220240 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:34.220244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:34.220804 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:34.220989 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.220994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.221728 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.221729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.222526 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.222529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.222943 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.222945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.223068 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.223497 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:00:34.223504 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:34.224847 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:34.226220 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:34.226228 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:34.226230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:34.226448 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:34.226662 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:34.226663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:34.226663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:34.226783 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:34.227047 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:34.227054 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:34.227188 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 13:00:34.227299 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 13:00:34.227449 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260806 13:00:34.227854 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 12131
I20260806 13:00:34.228001 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 13:00:34.228222 24908 external_mini_cluster.cc:1428] Setting key 79cd286fdc5c62c5db834c841ad1e6d6
I20260806 13:00:34.228327 12271 heartbeater.cc:347] Connected to a master server at 127.24.83.62:45117
I20260806 13:00:34.228444 12271 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:34.228641 12271 heartbeater.cc:511] Master 127.24.83.62:45117 requested a full tablet report, sending...
I20260806 13:00:34.229169 12066 ts_manager.cc:194] Registered new tserver with Master: fcd8076128424c66a2732d79058f4112 (127.24.83.1:44729)
I20260806 13:00:34.229779 12066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.1@KRBTEST.COM'} at 127.24.83.1:56903
WARNING: no policy specified for kudu/127.24.83.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
Entry for principal kudu/127.24.83.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab.
I20260806 13:00:34.252849 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45117
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:45627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:34.333830 12276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:34.334034 12276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:34.334057 12276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:34.335206 12276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:34.335250 12276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:34.335307 12276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 13:00:34.336746 12276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:45627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.2.keytab
--principal=kudu/127.24.83.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:34.336994 12276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:34.337224 12276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:34.342197 12283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:34.342391 12276 server_base.cc:1034] running on GCE node
W20260806 13:00:34.342204 12285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:34.342206 12282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:34.342684 12276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:34.342962 12276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:34.344141 12276 hybrid_clock.cc:648] HybridClock initialized: now 1786021234344115 us; error 34 us; skew 500 ppm
Aug 06 13:00:34 dist-test-slave-zkpd krb5kdc[12021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021234, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:34.346575 12276 init.cc:377] Logged in from keytab as kudu/127.24.83.2@KRBTEST.COM (short username kudu)
I20260806 13:00:34.347057 12276 webserver.cc:460] Webserver started at http://127.24.83.2:37825/ using document root <none> and password file <none>
I20260806 13:00:34.347283 12276 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:34.347352 12276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:34.347518 12276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:34.348587 12276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "42f7c3d305a580f686637485d7234221"
server_key_iv: "6700255334ca178a61b841a49d351fe1"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.348948 12276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "42f7c3d305a580f686637485d7234221"
server_key_iv: "6700255334ca178a61b841a49d351fe1"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.350246 12276 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:00:34.350951 12292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:34.351116 12276 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:34.351213 12276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "42f7c3d305a580f686637485d7234221"
server_key_iv: "6700255334ca178a61b841a49d351fe1"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.351295 12276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:34.368557 12276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:34.370366 12276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:34.370535 12276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:34.370774 12276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:34.371078 12276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:34.371130 12276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:34.371171 12276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:34.371199 12276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:34.376793 12276 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:38991
I20260806 13:00:34.376840 12408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:38991 every 8 connection(s)
I20260806 13:00:34.377130 12276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 13:00:34.377571 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 12276
I20260806 13:00:34.377694 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 13:00:34.377938 24908 external_mini_cluster.cc:1428] Setting key 68dde9f92f8faadcac495eaffd09680b
Aug 06 13:00:34 dist-test-slave-zkpd krb5kdc[12021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021234, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.2@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.83.3@KRBTEST.COM; defaulting to no policy
I20260806 13:00:34.386837 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.377828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:44793 (local address 127.24.83.62:45117)
0806 13:00:34.377871 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.377873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.378119 (+ 246us) server_negotiation.cc:407] Connection header received
0806 13:00:34.378821 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.378822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.378835 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.378881 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:34.379806 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.380390 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.381376 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.381590 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.381675 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.384025 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:34.384027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:34.384027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:34.384034 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:34.385428 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:34.385850 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:34.385851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:34.385852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:34.385915 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:34.386161 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:34.386163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:34.386163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:34.386275 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:34.386351 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.386630 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.386711 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:00:34.386993 12411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.377827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45117 (local address 127.24.83.2:44793)
0806 13:00:34.378001 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:34.378035 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:34.378717 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:34.378955 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:34.378959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:34.379523 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:34.379707 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.379712 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.380487 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.380488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.381280 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.381283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.381689 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.381690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.381820 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.382316 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:00:34.382324 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:34.383923 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:34.385546 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:34.385553 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:34.385555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:34.385772 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:34.385980 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:34.385980 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:34.385981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:34.386087 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:34.386342 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:34.386350 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:34.386488 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 13:00:34.386591 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 13:00:34.386746 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260806 13:00:34.387534 12409 heartbeater.cc:347] Connected to a master server at 127.24.83.62:45117
I20260806 13:00:34.387615 12409 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:34.387745 12409 heartbeater.cc:511] Master 127.24.83.62:45117 requested a full tablet report, sending...
I20260806 13:00:34.388171 12066 ts_manager.cc:194] Registered new tserver with Master: a2013a1e0d524d17bb0b8a8ee33adcfd (127.24.83.2:38991)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.3@KRBTEST.COM" created.
I20260806 13:00:34.388610 12066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.2@KRBTEST.COM'} at 127.24.83.2:44793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
Entry for principal kudu/127.24.83.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab.
I20260806 13:00:34.402912 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45117
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:45627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:34.484933 12414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:34.485126 12414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:34.485160 12414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:34.486240 12414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:34.486275 12414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:34.486301 12414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 13:00:34.487747 12414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:45627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.3.keytab
--principal=kudu/127.24.83.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:45117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:34.487984 12414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:34.488165 12414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:34.492900 12419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:34.492930 12420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:34.492882 12422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:34.493129 12414 server_base.cc:1034] running on GCE node
I20260806 13:00:34.493381 12414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:34.493655 12414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:34.494838 12414 hybrid_clock.cc:648] HybridClock initialized: now 1786021234494823 us; error 28 us; skew 500 ppm
Aug 06 13:00:34 dist-test-slave-zkpd krb5kdc[12021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021234, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:34.497414 12414 init.cc:377] Logged in from keytab as kudu/127.24.83.3@KRBTEST.COM (short username kudu)
I20260806 13:00:34.497828 12414 webserver.cc:460] Webserver started at http://127.24.83.3:36819/ using document root <none> and password file <none>
I20260806 13:00:34.498063 12414 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:34.498131 12414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:34.498272 12414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:34.499272 12414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "809ecf3c5919491c8268cd767856438d"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "5509ad209ad01250a82df2d7df5d1e85"
server_key_iv: "a47ac3b6fc2657371f4497254693bc9f"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.499639 12414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "809ecf3c5919491c8268cd767856438d"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "5509ad209ad01250a82df2d7df5d1e85"
server_key_iv: "a47ac3b6fc2657371f4497254693bc9f"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.501027 12414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:00:34.501745 12429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:34.501942 12414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:34.502040 12414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "809ecf3c5919491c8268cd767856438d"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "5509ad209ad01250a82df2d7df5d1e85"
server_key_iv: "a47ac3b6fc2657371f4497254693bc9f"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.502127 12414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:34.515055 12414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:34.517438 12414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:34.517596 12414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:34.517807 12414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:34.518105 12414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:34.518155 12414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:34.518195 12414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:34.518224 12414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:34.523867 12414 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:35167
I20260806 13:00:34.523965 12549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:35167 every 8 connection(s)
I20260806 13:00:34.524253 12414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 13:00:34.527699 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 12414
I20260806 13:00:34.527814 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 13:00:34.528024 24908 external_mini_cluster.cc:1428] Setting key 7f23870ab0fa387a8207d8fdf57734af
Aug 06 13:00:34 dist-test-slave-zkpd krb5kdc[12021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021234, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.3@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:34.533293 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.524964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:53597 (local address 127.24.83.62:45117)
0806 13:00:34.525035 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.525037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.525216 (+ 179us) server_negotiation.cc:407] Connection header received
0806 13:00:34.525668 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.525669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.525679 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.525730 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:34.526622 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.527109 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.528421 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.528704 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.528780 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.530821 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:34.530823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:34.530823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:34.530829 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:34.532087 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:34.532526 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:34.532527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:34.532528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:34.532573 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:34.532777 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:34.532777 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:34.532777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:34.532870 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:34.532926 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.533171 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.533232 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 13:00:34.533469 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.524950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:45117 (local address 127.24.83.3:53597)
0806 13:00:34.525110 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:34.525146 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:34.525586 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:34.525825 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:34.525829 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:34.526365 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:34.526546 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.526550 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.527186 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.527186 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.528309 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.528312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.528803 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.528804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.528935 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.529434 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:00:34.529442 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:34.530739 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:34.532223 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:34.532229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:34.532231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:34.532460 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:34.532631 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:00:34.532632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:00:34.532632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:00:34.532726 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:00:34.532935 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:34.532939 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:34.533065 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 13:00:34.533144 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 13:00:34.533260 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260806 13:00:34.533979 12550 heartbeater.cc:347] Connected to a master server at 127.24.83.62:45117
I20260806 13:00:34.534066 12550 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:34.534204 12550 heartbeater.cc:511] Master 127.24.83.62:45117 requested a full tablet report, sending...
I20260806 13:00:34.534559 12066 ts_manager.cc:194] Registered new tserver with Master: 809ecf3c5919491c8268cd767856438d (127.24.83.3:35167)
I20260806 13:00:34.534883 12066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.83.3@KRBTEST.COM'} at 127.24.83.3:53597
I20260806 13:00:34.539925 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:00:34.547086 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.541357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44310 (local address 127.24.83.62:45117)
0806 13:00:34.541411 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.541413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.541562 (+ 149us) server_negotiation.cc:407] Connection header received
0806 13:00:34.541641 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.541642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.541651 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.541706 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:34.542253 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.542722 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.543462 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.543663 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.543738 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.544864 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:34.544866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:34.544866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:34.544873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:34.545956 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:34.546287 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:34.546288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:34.546289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:34.546346 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:34.546558 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:34.546559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:34.546559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:34.546697 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:34.546807 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.546919 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.546992 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 13:00:34.548681 12066 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:00:34.550150 12066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:00:34.556672 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.552331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42820 (local address 127.24.83.2:38991)
0806 13:00:34.552541 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.552545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.552554 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:00:34.552591 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.552592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.552614 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.552706 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:34.552973 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.554293 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.555491 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.556233 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.556309 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.556362 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.556541 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.556589 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":84,"threads_started":1}
I20260806 13:00:34.556680 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.552216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:38991 (local address 127.0.0.1:42820)
0806 13:00:34.552345 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:34.552398 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:34.552463 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:34.552801 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:34.552804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:34.552807 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:34.552869 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.552871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.554402 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.554403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.555408 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.555410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.556324 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.556325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.556427 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.556442 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 13:00:34.556516 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:00:34.556587 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":29,"thread_start_us":68,"threads_started":1}
I20260806 13:00:34.556947 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.552524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.24.83.3:35167)
0806 13:00:34.552714 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.552719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.552832 (+ 113us) server_negotiation.cc:407] Connection header received
0806 13:00:34.552875 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.552880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.552898 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.552981 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:34.553139 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.553971 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.555567 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.556233 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.556305 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.556362 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.556832 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.556874 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260806 13:00:34.556679 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.552216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:35167 (local address 127.0.0.1:50462)
0806 13:00:34.552696 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:34.552707 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:34.552774 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:34.553051 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:34.553052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:34.553054 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:34.553086 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.553087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.554068 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.554069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.555489 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.555492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.556322 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.556323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.556427 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.556442 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 13:00:34.556516 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:00:34.556587 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":78,"thread_start_us":61,"threads_started":1}
I20260806 13:00:34.557602 12563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.552414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:44729 (local address 127.0.0.1:39626)
0806 13:00:34.552696 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:34.552707 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:34.552761 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:34.552966 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:34.552967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:34.552969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:34.553022 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.553023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.554382 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.554383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.555493 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.555495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.557375 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.557376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.557452 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.557456 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:00:34.557507 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:00:34.557545 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":2,"thread_start_us":46,"threads_started":1}
I20260806 13:00:34.557492 12341 tablet_service.cc:1467] Processing CreateTablet for tablet 0c695002ca9a4a5a95ec9bafbff51884 (DEFAULT_TABLE table=test-table [id=7fdcee90035a46d883322a3514c800f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:34.557776 12565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.552539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39626 (local address 127.24.83.1:44729)
0806 13:00:34.552810 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.552813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.552823 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:00:34.552860 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.552862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.552877 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.552958 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:34.553100 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.554276 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.556398 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.557130 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.557248 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.557311 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.557683 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.557719 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":82,"threads_started":1}
I20260806 13:00:34.557850 12341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c695002ca9a4a5a95ec9bafbff51884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:34.557996 12479 tablet_service.cc:1467] Processing CreateTablet for tablet 0c695002ca9a4a5a95ec9bafbff51884 (DEFAULT_TABLE table=test-table [id=7fdcee90035a46d883322a3514c800f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:34.558305 12479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c695002ca9a4a5a95ec9bafbff51884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:34.560673 12200 tablet_service.cc:1467] Processing CreateTablet for tablet 0c695002ca9a4a5a95ec9bafbff51884 (DEFAULT_TABLE table=test-table [id=7fdcee90035a46d883322a3514c800f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:34.560997 12200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c695002ca9a4a5a95ec9bafbff51884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:34.561466 12566 tablet_bootstrap.cc:492] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd: Bootstrap starting.
I20260806 13:00:34.561754 12567 tablet_bootstrap.cc:492] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d: Bootstrap starting.
I20260806 13:00:34.562192 12566 tablet_bootstrap.cc:654] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:34.562371 12567 tablet_bootstrap.cc:654] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:34.562769 12566 log.cc:824] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:34.562848 12567 log.cc:824] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:34.563537 12566 tablet_bootstrap.cc:492] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd: No bootstrap required, opened a new log
I20260806 13:00:34.563627 12566 ts_tablet_manager.cc:1397] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:00:34.564409 12571 tablet_bootstrap.cc:492] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112: Bootstrap starting.
I20260806 13:00:34.564913 12571 tablet_bootstrap.cc:654] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:34.565307 12571 log.cc:824] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:34.565299 12566 raft_consensus.cc:348] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ecf3c5919491c8268cd767856438d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35167 } } peers { permanent_uuid: "fcd8076128424c66a2732d79058f4112" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44729 } } peers { permanent_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 38991 } }
I20260806 13:00:34.565423 12566 raft_consensus.cc:374] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:34.565444 12566 raft_consensus.cc:729] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2013a1e0d524d17bb0b8a8ee33adcfd, State: Initialized, Role: FOLLOWER
I20260806 13:00:34.565541 12566 consensus_queue.cc:260] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ecf3c5919491c8268cd767856438d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35167 } } peers { permanent_uuid: "fcd8076128424c66a2732d79058f4112" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44729 } } peers { permanent_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 38991 } }
I20260806 13:00:34.565788 12566 ts_tablet_manager.cc:1428] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 13:00:34.565835 12409 heartbeater.cc:503] Master 127.24.83.62:45117 was elected leader, sending a full tablet report...
I20260806 13:00:34.566113 12571 tablet_bootstrap.cc:492] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112: No bootstrap required, opened a new log
I20260806 13:00:34.566203 12571 ts_tablet_manager.cc:1397] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:34.566480 12567 tablet_bootstrap.cc:492] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d: No bootstrap required, opened a new log
I20260806 13:00:34.566566 12567 ts_tablet_manager.cc:1397] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 13:00:34.567983 12571 raft_consensus.cc:348] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ecf3c5919491c8268cd767856438d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35167 } } peers { permanent_uuid: "fcd8076128424c66a2732d79058f4112" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44729 } } peers { permanent_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 38991 } }
I20260806 13:00:34.568146 12571 raft_consensus.cc:374] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:34.568223 12571 raft_consensus.cc:729] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcd8076128424c66a2732d79058f4112, State: Initialized, Role: FOLLOWER
I20260806 13:00:34.568284 12567 raft_consensus.cc:348] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ecf3c5919491c8268cd767856438d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35167 } } peers { permanent_uuid: "fcd8076128424c66a2732d79058f4112" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44729 } } peers { permanent_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 38991 } }
I20260806 13:00:34.568392 12571 consensus_queue.cc:260] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ecf3c5919491c8268cd767856438d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35167 } } peers { permanent_uuid: "fcd8076128424c66a2732d79058f4112" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44729 } } peers { permanent_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 38991 } }
I20260806 13:00:34.568459 12567 raft_consensus.cc:374] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:34.568512 12567 raft_consensus.cc:729] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809ecf3c5919491c8268cd767856438d, State: Initialized, Role: FOLLOWER
I20260806 13:00:34.568620 12567 consensus_queue.cc:260] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ecf3c5919491c8268cd767856438d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35167 } } peers { permanent_uuid: "fcd8076128424c66a2732d79058f4112" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44729 } } peers { permanent_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 38991 } }
I20260806 13:00:34.568660 12571 ts_tablet_manager.cc:1428] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:00:34.568692 12271 heartbeater.cc:503] Master 127.24.83.62:45117 was elected leader, sending a full tablet report...
I20260806 13:00:34.568886 12567 ts_tablet_manager.cc:1428] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:00:34.568992 12550 heartbeater.cc:503] Master 127.24.83.62:45117 was elected leader, sending a full tablet report...
W20260806 13:00:34.627662 12410 tablet.cc:2369] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:00:34.715169 12573 raft_consensus.cc:482] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:00:34.715305 12573 raft_consensus.cc:504] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ecf3c5919491c8268cd767856438d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35167 } } peers { permanent_uuid: "fcd8076128424c66a2732d79058f4112" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44729 } } peers { permanent_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 38991 } }
I20260806 13:00:34.715724 12573 leader_election.cc:288] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 809ecf3c5919491c8268cd767856438d (127.24.83.3:35167), fcd8076128424c66a2732d79058f4112 (127.24.83.1:44729)
I20260806 13:00:34.718551 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.716029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:37661 (local address 127.24.83.3:35167)
0806 13:00:34.716088 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.716091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.716105 (+ 14us) server_negotiation.cc:407] Connection header received
0806 13:00:34.716139 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.716140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.716151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.716225 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:34.716405 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.716887 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.717798 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.718186 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.718267 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.718278 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.718443 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.718485 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 13:00:34.718550 12411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.715965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:35167 (local address 127.24.83.2:37661)
0806 13:00:34.716022 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:34.716034 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:34.716079 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:34.716246 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:34.716247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:34.716249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:34.716324 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.716325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.716968 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.716969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.717701 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.717704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.718278 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.718279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.718366 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.718369 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:34.718436 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 13:00:34.718477 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 13:00:34.718812 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.716117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:44729 (local address 127.24.83.2:37953)
0806 13:00:34.716292 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:34.716300 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:34.716376 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:34.716524 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:34.716525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:34.716526 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:34.716625 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.716627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.717236 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.717237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.717969 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:34.717971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.718528 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.718529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:34.718608 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.718614 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 13:00:34.718671 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 13:00:34.718722 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":119,"threads_started":1}
I20260806 13:00:34.718827 12565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.716175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:37953 (local address 127.24.83.1:44729)
0806 13:00:34.716241 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.716244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.716366 (+ 122us) server_negotiation.cc:407] Connection header received
0806 13:00:34.716430 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.716431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.716443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.716508 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:34.716691 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.717149 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.718063 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.718440 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.718524 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.718534 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.718714 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.718760 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 13:00:34.718897 12499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c695002ca9a4a5a95ec9bafbff51884" candidate_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809ecf3c5919491c8268cd767856438d" is_pre_election: true
I20260806 13:00:34.719053 12499 raft_consensus.cc:2454] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2013a1e0d524d17bb0b8a8ee33adcfd in term 0.
W20260806 13:00:34.719277 12272 tablet.cc:2369] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:00:34.719296 12293 leader_election.cc:302] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 809ecf3c5919491c8268cd767856438d, a2013a1e0d524d17bb0b8a8ee33adcfd; no voters:
I20260806 13:00:34.719295 12223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c695002ca9a4a5a95ec9bafbff51884" candidate_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcd8076128424c66a2732d79058f4112" is_pre_election: true
I20260806 13:00:34.719488 12223 raft_consensus.cc:2454] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2013a1e0d524d17bb0b8a8ee33adcfd in term 0.
I20260806 13:00:34.719481 12573 raft_consensus.cc:2781] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:00:34.719542 12573 raft_consensus.cc:482] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:00:34.719568 12573 raft_consensus.cc:3037] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:34.720449 12573 raft_consensus.cc:504] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ecf3c5919491c8268cd767856438d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35167 } } peers { permanent_uuid: "fcd8076128424c66a2732d79058f4112" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44729 } } peers { permanent_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 38991 } }
I20260806 13:00:34.720582 12573 leader_election.cc:288] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [CANDIDATE]: Term 1 election: Requested vote from peers 809ecf3c5919491c8268cd767856438d (127.24.83.3:35167), fcd8076128424c66a2732d79058f4112 (127.24.83.1:44729)
I20260806 13:00:34.720738 12499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c695002ca9a4a5a95ec9bafbff51884" candidate_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809ecf3c5919491c8268cd767856438d"
I20260806 13:00:34.720755 12223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c695002ca9a4a5a95ec9bafbff51884" candidate_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcd8076128424c66a2732d79058f4112"
I20260806 13:00:34.720839 12223 raft_consensus.cc:3037] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:34.720844 12499 raft_consensus.cc:3037] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:34.721547 12223 raft_consensus.cc:2454] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2013a1e0d524d17bb0b8a8ee33adcfd in term 1.
I20260806 13:00:34.721546 12499 raft_consensus.cc:2454] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2013a1e0d524d17bb0b8a8ee33adcfd in term 1.
I20260806 13:00:34.721720 12293 leader_election.cc:302] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 809ecf3c5919491c8268cd767856438d, a2013a1e0d524d17bb0b8a8ee33adcfd; no voters:
I20260806 13:00:34.721905 12573 raft_consensus.cc:2781] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:34.722079 12573 raft_consensus.cc:686] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 1 LEADER]: Becoming Leader. State: Replica: a2013a1e0d524d17bb0b8a8ee33adcfd, State: Running, Role: LEADER
I20260806 13:00:34.722175 12573 consensus_queue.cc:237] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ecf3c5919491c8268cd767856438d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35167 } } peers { permanent_uuid: "fcd8076128424c66a2732d79058f4112" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44729 } } peers { permanent_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 38991 } }
I20260806 13:00:34.723042 12066 catalog_manager.cc:5495] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd reported cstate change: term changed from 0 to 1, leader changed from <none> to a2013a1e0d524d17bb0b8a8ee33adcfd (127.24.83.2). New cstate: current_term: 1 leader_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ecf3c5919491c8268cd767856438d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcd8076128424c66a2732d79058f4112" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2013a1e0d524d17bb0b8a8ee33adcfd" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 38991 } health_report { overall_health: HEALTHY } } }
I20260806 13:00:34.735983 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.733745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42822 (local address 127.24.83.2:38991)
0806 13:00:34.733806 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.733808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.733822 (+ 14us) server_negotiation.cc:407] Connection header received
0806 13:00:34.733878 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.733878 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.733895 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.733954 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:00:34.734060 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.734490 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.735238 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.735436 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.735499 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.735638 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:00:34.735741 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:00:34.735801 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.735836 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.735853 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 13:00:34.737834 12499 raft_consensus.cc:1261] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d [term 1 FOLLOWER]: Refusing update from remote peer a2013a1e0d524d17bb0b8a8ee33adcfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:34.737982 12223 raft_consensus.cc:1261] T 0c695002ca9a4a5a95ec9bafbff51884 P fcd8076128424c66a2732d79058f4112 [term 1 FOLLOWER]: Refusing update from remote peer a2013a1e0d524d17bb0b8a8ee33adcfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:34.738193 12573 consensus_queue.cc:1035] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "809ecf3c5919491c8268cd767856438d" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:34.738358 12578 consensus_queue.cc:1035] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcd8076128424c66a2732d79058f4112" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 44729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:34.741266 12582 mvcc.cc:205] Tried to move back new op lower bound from 7315542977483100160 to 7315542977422696448. Current Snapshot: MvccSnapshot[applied={T|T < 7315542977483100160}]
I20260806 13:00:34.742871 12066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:34.742969 12066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:34.743755 12066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05be91c70a4b4d5c946e1cf75ae777cf: Sending DeleteTablet for 3 replicas of tablet 0c695002ca9a4a5a95ec9bafbff51884
I20260806 13:00:34.744103 12341 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c695002ca9a4a5a95ec9bafbff51884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:34 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:42820
I20260806 13:00:34.744120 12200 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c695002ca9a4a5a95ec9bafbff51884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:34 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:39626
I20260806 13:00:34.744155 12479 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c695002ca9a4a5a95ec9bafbff51884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:34 UTC) from {username='kudu', principal='kudu/127.24.83.62@KRBTEST.COM'} at 127.0.0.1:50462
I20260806 13:00:34.744450 12594 tablet_replica.cc:330] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d: stopping tablet replica
I20260806 13:00:34.744495 12592 tablet_replica.cc:330] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd: stopping tablet replica
I20260806 13:00:34.744591 12594 raft_consensus.cc:2229] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:00:34.744648 12592 raft_consensus.cc:2229] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:00:34.744733 12594 raft_consensus.cc:2258] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:34.744768 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 12131
I20260806 13:00:34.745464 12592 raft_consensus.cc:2258] T 0c695002ca9a4a5a95ec9bafbff51884 P a2013a1e0d524d17bb0b8a8ee33adcfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:34.746136 12594 ts_tablet_manager.cc:1905] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:00:34.748009 12594 ts_tablet_manager.cc:1918] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:00:34.748080 12594 log.cc:1196] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/0c695002ca9a4a5a95ec9bafbff51884
I20260806 13:00:34.748365 12594 ts_tablet_manager.cc:1939] T 0c695002ca9a4a5a95ec9bafbff51884 P 809ecf3c5919491c8268cd767856438d: Deleting consensus metadata
I20260806 13:00:34.748910 12050 catalog_manager.cc:4841] TS 809ecf3c5919491c8268cd767856438d (127.24.83.3:35167): tablet 0c695002ca9a4a5a95ec9bafbff51884 (table test-table [id=7fdcee90035a46d883322a3514c800f4]) successfully deleted
W20260806 13:00:34.751407 12050 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:44729 (error 108)
I20260806 13:00:34.751775 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 12276
I20260806 13:00:34.752117 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.751975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:44729 (local address 127.0.0.1:39636)
0806 13:00:34.752037 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:34.752080 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 13:00:34.752229 12050 catalog_manager.cc:4584] TS fcd8076128424c66a2732d79058f4112 (127.24.83.1:44729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c695002ca9a4a5a95ec9bafbff51884: Network error: Client connection negotiation failed: client connection to 127.24.83.1:44729: connect: Connection refused (error 111)
I20260806 13:00:34.757853 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 12414
I20260806 13:00:34.757953 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.757817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:38991 (local address 127.0.0.1:42838)
0806 13:00:34.757873 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:34.757899 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.2:38991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 13:00:34.764823 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 12034
2026-08-06T13:00:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2015 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T13:00:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:34Z Disabled control of system clock
I20260806 13:00:34.792798 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:39095
--webserver_interface=127.24.83.62
--webserver_port=0
--builtin_ntp_servers=127.24.83.20:34521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:39095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:00:34.873962 12599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:34.874161 12599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:34.874181 12599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:34.875304 12599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:34.875344 12599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:34.875357 12599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:34.875394 12599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:34.875422 12599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:34.876840 12599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:39095
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:39095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:34.877077 12599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:34.877362 12599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:34.881996 12605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:34.882004 12607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:34.882014 12604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:34.882184 12599 server_base.cc:1034] running on GCE node
I20260806 13:00:34.882441 12599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:34.882696 12599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:34.883826 12599 hybrid_clock.cc:648] HybridClock initialized: now 1786021234883809 us; error 33 us; skew 500 ppm
I20260806 13:00:34.885093 12599 webserver.cc:460] Webserver started at http://127.24.83.62:37341/ using document root <none> and password file <none>
I20260806 13:00:34.885342 12599 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:34.885401 12599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:34.885525 12599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:34.886579 12599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "7e563d6951b44991b1d787bd444e5ec8"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "18080bf3726a9fa31857cb11eb353820"
server_key_iv: "9c3548548da874715be2fcdbee8421a2"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.886930 12599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "7e563d6951b44991b1d787bd444e5ec8"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "18080bf3726a9fa31857cb11eb353820"
server_key_iv: "9c3548548da874715be2fcdbee8421a2"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.888186 12599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:00:34.888890 12614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:34.889072 12599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:00:34.889159 12599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "7e563d6951b44991b1d787bd444e5ec8"
format_stamp: "Formatted at 2026-08-06 13:00:34 on dist-test-slave-zkpd"
server_key: "18080bf3726a9fa31857cb11eb353820"
server_key_iv: "9c3548548da874715be2fcdbee8421a2"
server_key_version: "encryptionkey@0"
I20260806 13:00:34.889230 12599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:34.915149 12599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:34.915464 12599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:34.915613 12599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:34.919731 12599 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:39095
I20260806 13:00:34.919807 12668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:39095 every 8 connection(s)
I20260806 13:00:34.920140 12599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:34.921075 12669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:34.923297 12669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8: Bootstrap starting.
I20260806 13:00:34.923866 12669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:34.924247 12669 log.cc:824] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:34.924907 12669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8: No bootstrap required, opened a new log
I20260806 13:00:34.926450 12669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e563d6951b44991b1d787bd444e5ec8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 39095 } }
I20260806 13:00:34.926580 12669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:34.926627 12669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e563d6951b44991b1d787bd444e5ec8, State: Initialized, Role: FOLLOWER
I20260806 13:00:34.926772 12669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e563d6951b44991b1d787bd444e5ec8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 39095 } }
I20260806 13:00:34.926834 12669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:34.926856 12669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:34.926918 12669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:34.927078 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 12599
I20260806 13:00:34.927201 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:34.927377 24908 external_mini_cluster.cc:1428] Setting key 322221d95840b589327de13bc11f120a
I20260806 13:00:34.927634 12669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e563d6951b44991b1d787bd444e5ec8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 39095 } }
I20260806 13:00:34.927757 12669 leader_election.cc:302] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e563d6951b44991b1d787bd444e5ec8; no voters:
I20260806 13:00:34.927950 12669 leader_election.cc:288] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:34.928042 12673 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:34.928194 12669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:34.928299 12673 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [term 1 LEADER]: Becoming Leader. State: Replica: 7e563d6951b44991b1d787bd444e5ec8, State: Running, Role: LEADER
I20260806 13:00:34.928426 12673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e563d6951b44991b1d787bd444e5ec8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 39095 } }
I20260806 13:00:34.928856 12675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e563d6951b44991b1d787bd444e5ec8. Latest consensus state: current_term: 1 leader_uuid: "7e563d6951b44991b1d787bd444e5ec8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e563d6951b44991b1d787bd444e5ec8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 39095 } } }
I20260806 13:00:34.928964 12675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:34.929185 12683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:34.929422 12676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e563d6951b44991b1d787bd444e5ec8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e563d6951b44991b1d787bd444e5ec8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 39095 } } }
I20260806 13:00:34.929505 12676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:34.929811 12683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:34.932003 12683 catalog_manager.cc:1347] Generated new cluster ID: 844edefaba1a4f3697056ed366f60592
I20260806 13:00:34.932065 12683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:34.933668 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:34.927828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41354 (local address 127.24.83.62:39095)
0806 13:00:34.928073 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:34.928082 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:34.928115 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:00:34.928226 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:34.928233 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:34.928328 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:34.928491 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:34.928746 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.930502 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.931558 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:34.932190 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:34.932318 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:34.932976 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:34.932980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:34.932980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:34.933178 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:34.933229 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:34.933243 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:34.933316 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:34.933380 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 13:00:34.933462 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":96,"threads_started":1}
I20260806 13:00:34.937381 12683 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:34.938057 12683 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:34.947973 12683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8: Generated new TSK 0
I20260806 13:00:34.948150 12683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:34.957159 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.1:0
--local_ip_for_outbound_sockets=127.24.83.1
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:39095
--builtin_ntp_servers=127.24.83.20:34521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:00:35.040297 12693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:35.040521 12693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:35.040561 12693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:35.041826 12693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:35.041882 12693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:35.041925 12693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.1
I20260806 13:00:35.043287 12693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.83.1
--webserver_port=0
--tserver_master_addrs=127.24.83.62:39095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.1
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:35.043548 12693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:35.043870 12693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:35.049641 12699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:35.049952 12693 server_base.cc:1034] running on GCE node
W20260806 13:00:35.049997 12701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:35.050071 12698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:35.050529 12693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:35.050791 12693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:35.051965 12693 hybrid_clock.cc:648] HybridClock initialized: now 1786021235051951 us; error 48 us; skew 500 ppm
I20260806 13:00:35.053414 12693 webserver.cc:460] Webserver started at http://127.24.83.1:42407/ using document root <none> and password file <none>
I20260806 13:00:35.053653 12693 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:35.053722 12693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:35.053846 12693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:35.055176 12693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/data/instance:
uuid: "3429f5af490d457e8ebe430b1d5c10c1"
format_stamp: "Formatted at 2026-08-06 13:00:35 on dist-test-slave-zkpd"
server_key: "d2fab668051fd82f19031c9ab49ec410"
server_key_iv: "32bc378fb4b750bd3962bce69fe6e0da"
server_key_version: "encryptionkey@0"
I20260806 13:00:35.055559 12693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance:
uuid: "3429f5af490d457e8ebe430b1d5c10c1"
format_stamp: "Formatted at 2026-08-06 13:00:35 on dist-test-slave-zkpd"
server_key: "d2fab668051fd82f19031c9ab49ec410"
server_key_iv: "32bc378fb4b750bd3962bce69fe6e0da"
server_key_version: "encryptionkey@0"
I20260806 13:00:35.057201 12693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:00:35.058091 12708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:35.058281 12693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:35.058369 12693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/wal
uuid: "3429f5af490d457e8ebe430b1d5c10c1"
format_stamp: "Formatted at 2026-08-06 13:00:35 on dist-test-slave-zkpd"
server_key: "d2fab668051fd82f19031c9ab49ec410"
server_key_iv: "32bc378fb4b750bd3962bce69fe6e0da"
server_key_version: "encryptionkey@0"
I20260806 13:00:35.058439 12693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:35.073570 12693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:35.073863 12693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:35.073990 12693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:35.074201 12693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:35.074520 12693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:35.074556 12693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:35.074581 12693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:35.074599 12693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:35.081766 12693 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.1:34671
I20260806 13:00:35.082217 12693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/data/info.pb
I20260806 13:00:35.082671 12824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.1:34671 every 8 connection(s)
I20260806 13:00:35.088721 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.082880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.1:38203 (local address 127.24.83.62:39095)
0806 13:00:35.082934 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:35.082936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:35.083253 (+ 317us) server_negotiation.cc:407] Connection header received
0806 13:00:35.084043 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:35.084045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:35.084058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:35.084090 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:35.085039 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.085893 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.087357 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.087553 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.087615 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.088486 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:35.088487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:35.088487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:35.088491 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:35.088505 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:35.088508 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:35.088552 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:35.088637 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 13:00:35.088663 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 13:00:35.088932 12827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.082902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:39095 (local address 127.24.83.1:38203)
0806 13:00:35.083111 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:35.083144 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:35.083914 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:35.084260 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:35.084264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:35.084274 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:35.084932 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.084938 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.086002 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.086003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.087264 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.087267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.087643 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.087644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.087788 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.088341 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:00:35.088342 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:35.088377 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:00:35.088401 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:35.088566 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:35.088569 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:35.088572 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:35.088631 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:00:35.088723 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260806 13:00:35.089389 12825 heartbeater.cc:347] Connected to a master server at 127.24.83.62:39095
I20260806 13:00:35.089476 12825 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:35.089661 12825 heartbeater.cc:511] Master 127.24.83.62:39095 requested a full tablet report, sending...
I20260806 13:00:35.090152 12632 ts_manager.cc:194] Registered new tserver with Master: 3429f5af490d457e8ebe430b1d5c10c1 (127.24.83.1:34671)
I20260806 13:00:35.090742 12632 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.1:38203
I20260806 13:00:35.092415 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 12693
I20260806 13:00:35.092538 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-0/wal/instance
I20260806 13:00:35.092751 24908 external_mini_cluster.cc:1428] Setting key f8d09c422f35f205332936b09eb4ee3a
I20260806 13:00:35.093755 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.2:0
--local_ip_for_outbound_sockets=127.24.83.2
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:39095
--builtin_ntp_servers=127.24.83.20:34521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:00:35.175263 12828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:35.175494 12828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:35.175532 12828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:35.176744 12828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:35.176812 12828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:35.176851 12828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.2
I20260806 13:00:35.178174 12828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.83.2
--webserver_port=0
--tserver_master_addrs=127.24.83.62:39095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.2
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:35.178447 12828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:35.178746 12828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:35.183610 12833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:35.183624 12834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:35.183630 12836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:35.183735 12828 server_base.cc:1034] running on GCE node
I20260806 13:00:35.184135 12828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:35.184381 12828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:35.185540 12828 hybrid_clock.cc:648] HybridClock initialized: now 1786021235185523 us; error 46 us; skew 500 ppm
I20260806 13:00:35.186846 12828 webserver.cc:460] Webserver started at http://127.24.83.2:41181/ using document root <none> and password file <none>
I20260806 13:00:35.187099 12828 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:35.187170 12828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:35.187319 12828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:35.188418 12828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/data/instance:
uuid: "21e264971d1045d7a53eae25a0ae6403"
format_stamp: "Formatted at 2026-08-06 13:00:35 on dist-test-slave-zkpd"
server_key: "bd5e3c94ca8a14ce211e9b410c273de0"
server_key_iv: "ef4f563bc91706b41f935c1779fb642b"
server_key_version: "encryptionkey@0"
I20260806 13:00:35.188786 12828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance:
uuid: "21e264971d1045d7a53eae25a0ae6403"
format_stamp: "Formatted at 2026-08-06 13:00:35 on dist-test-slave-zkpd"
server_key: "bd5e3c94ca8a14ce211e9b410c273de0"
server_key_iv: "ef4f563bc91706b41f935c1779fb642b"
server_key_version: "encryptionkey@0"
I20260806 13:00:35.190056 12828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:00:35.190738 12842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:35.190905 12828 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:35.191008 12828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/wal
uuid: "21e264971d1045d7a53eae25a0ae6403"
format_stamp: "Formatted at 2026-08-06 13:00:35 on dist-test-slave-zkpd"
server_key: "bd5e3c94ca8a14ce211e9b410c273de0"
server_key_iv: "ef4f563bc91706b41f935c1779fb642b"
server_key_version: "encryptionkey@0"
I20260806 13:00:35.191085 12828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:35.199782 12828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:35.200093 12828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:35.200243 12828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:35.200459 12828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:35.200780 12828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:35.200834 12828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:35.200865 12828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:35.200891 12828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:35.207726 12828 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.2:35669
I20260806 13:00:35.207783 12958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.2:35669 every 8 connection(s)
I20260806 13:00:35.208161 12828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/data/info.pb
I20260806 13:00:35.209156 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 12828
I20260806 13:00:35.209278 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/wal/instance
I20260806 13:00:35.209482 24908 external_mini_cluster.cc:1428] Setting key 977416bee0a03ee40b34b16b260d17ca
I20260806 13:00:35.210453 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.83.3:0
--local_ip_for_outbound_sockets=127.24.83.3
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:39095
--builtin_ntp_servers=127.24.83.20:34521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 13:00:35.214015 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.208824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.2:57381 (local address 127.24.83.62:39095)
0806 13:00:35.208869 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:35.208871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:35.209182 (+ 311us) server_negotiation.cc:407] Connection header received
0806 13:00:35.209874 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:35.209875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:35.209899 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:35.209951 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:35.210843 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.211314 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.212574 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.212783 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.212881 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.213737 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:35.213739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:35.213739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:35.213744 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:35.213761 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:35.213765 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:35.213818 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:35.213905 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 13:00:35.213933 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:00:35.214218 12961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.208827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:39095 (local address 127.24.83.2:57381)
0806 13:00:35.209025 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:35.209070 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:35.209756 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:35.210027 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:35.210031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:35.210042 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:35.210752 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.210759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.211412 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.211413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.212471 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.212474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.212888 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.212889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.213042 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.213601 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:00:35.213601 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:35.213629 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:00:35.213650 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:35.213824 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:35.213829 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:35.213832 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:35.213891 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:00:35.213987 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260806 13:00:35.214627 12959 heartbeater.cc:347] Connected to a master server at 127.24.83.62:39095
I20260806 13:00:35.214732 12959 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:35.214910 12959 heartbeater.cc:511] Master 127.24.83.62:39095 requested a full tablet report, sending...
I20260806 13:00:35.215265 12632 ts_manager.cc:194] Registered new tserver with Master: 21e264971d1045d7a53eae25a0ae6403 (127.24.83.2:35669)
I20260806 13:00:35.215641 12632 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.2:57381
W20260806 13:00:35.291782 12962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:35.292014 12962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:35.292053 12962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:35.293268 12962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:35.293327 12962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:35.293371 12962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.83.3
I20260806 13:00:35.294828 12962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.83.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.83.3
--webserver_port=0
--tserver_master_addrs=127.24.83.62:39095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.83.3
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:35.295082 12962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:35.295377 12962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:35.300257 12967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:35.300239 12968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:35.300299 12962 server_base.cc:1034] running on GCE node
W20260806 13:00:35.300230 12970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:35.300602 12962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:35.300879 12962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:35.302048 12962 hybrid_clock.cc:648] HybridClock initialized: now 1786021235302034 us; error 43 us; skew 500 ppm
I20260806 13:00:35.303395 12962 webserver.cc:460] Webserver started at http://127.24.83.3:41773/ using document root <none> and password file <none>
I20260806 13:00:35.303637 12962 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:35.303709 12962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:35.303925 12962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:35.304972 12962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/data/instance:
uuid: "502bfcad4d1042939aea89179d57d258"
format_stamp: "Formatted at 2026-08-06 13:00:35 on dist-test-slave-zkpd"
server_key: "7bcd636a34211a6823bfe8547f30b94b"
server_key_iv: "46330843d0e0564f82f67ebe6529085e"
server_key_version: "encryptionkey@0"
I20260806 13:00:35.305354 12962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance:
uuid: "502bfcad4d1042939aea89179d57d258"
format_stamp: "Formatted at 2026-08-06 13:00:35 on dist-test-slave-zkpd"
server_key: "7bcd636a34211a6823bfe8547f30b94b"
server_key_iv: "46330843d0e0564f82f67ebe6529085e"
server_key_version: "encryptionkey@0"
I20260806 13:00:35.306666 12962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:00:35.307675 12976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:35.308028 12962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:35.308143 12962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/wal
uuid: "502bfcad4d1042939aea89179d57d258"
format_stamp: "Formatted at 2026-08-06 13:00:35 on dist-test-slave-zkpd"
server_key: "7bcd636a34211a6823bfe8547f30b94b"
server_key_iv: "46330843d0e0564f82f67ebe6529085e"
server_key_version: "encryptionkey@0"
I20260806 13:00:35.308221 12962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:35.320179 12962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:35.320477 12962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:35.320633 12962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:35.320859 12962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:00:35.321184 12962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:00:35.321238 12962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:35.321285 12962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:00:35.321312 12962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:35.326869 12962 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.3:40039
I20260806 13:00:35.326941 13092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.3:40039 every 8 connection(s)
I20260806 13:00:35.327278 12962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/data/info.pb
I20260806 13:00:35.332728 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.328098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:42637 (local address 127.24.83.62:39095)
0806 13:00:35.328149 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:35.328151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:35.328501 (+ 350us) server_negotiation.cc:407] Connection header received
0806 13:00:35.329045 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:35.329046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:35.329058 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:35.329122 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:35.329857 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.330352 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.331322 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.331515 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.331571 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.332503 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:35.332504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:35.332504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:35.332508 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:35.332521 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:35.332524 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:35.332567 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:35.332644 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 13:00:35.332668 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:00:35.332950 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.328127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:39095 (local address 127.24.83.3:42637)
0806 13:00:35.328359 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:35.328405 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:35.328905 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:35.329198 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:35.329201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:35.329209 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:00:35.329780 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.329785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.330436 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.330437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.331237 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.331239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.331619 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.331620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.331741 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.332380 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:00:35.332381 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:00:35.332409 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:00:35.332431 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:00:35.332580 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:00:35.332585 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:00:35.332587 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:00:35.332636 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 13:00:35.332720 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
I20260806 13:00:35.333421 13093 heartbeater.cc:347] Connected to a master server at 127.24.83.62:39095
I20260806 13:00:35.333528 13093 heartbeater.cc:464] Registering TS with master...
I20260806 13:00:35.333732 13093 heartbeater.cc:511] Master 127.24.83.62:39095 requested a full tablet report, sending...
I20260806 13:00:35.334102 12632 ts_manager.cc:194] Registered new tserver with Master: 502bfcad4d1042939aea89179d57d258 (127.24.83.3:40039)
I20260806 13:00:35.334473 12632 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.83.3:42637
I20260806 13:00:35.335731 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 12962
I20260806 13:00:35.335840 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/wal/instance
I20260806 13:00:35.336064 24908 external_mini_cluster.cc:1428] Setting key 51e749401e0b30420995c27e551a9361
I20260806 13:00:35.336872 24908 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:00:35.341161 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.338505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.24.83.62:39095)
0806 13:00:35.338547 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:35.338549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:35.338650 (+ 101us) server_negotiation.cc:407] Connection header received
0806 13:00:35.338684 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:35.338685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:35.338693 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:35.338742 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:35.338891 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.339331 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.340099 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.340270 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.340321 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.340959 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:35.340960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:35.340960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:35.340963 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:35.340976 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:35.340979 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:35.341016 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:35.341049 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 13:00:35.341064 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:00:35.342578 12632 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:00:35.344127 12632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:00:35.350456 13106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.346654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.24.83.2:35669)
0806 13:00:35.346834 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:35.346840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:35.346850 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:00:35.346896 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:35.346898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:35.346914 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:35.347113 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:35.347171 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.348206 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.349282 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.350042 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.350120 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.350217 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:35.350327 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 13:00:35.350373 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260806 13:00:35.350457 13102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.346193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:35669 (local address 127.0.0.1:34422)
0806 13:00:35.346596 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:35.346605 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:35.346646 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:35.347013 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:35.347014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:35.347015 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:35.347045 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.347046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.348290 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.348291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.349190 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.349192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.350128 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.350128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.350185 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.350197 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 13:00:35.350301 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 13:00:35.350361 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":84,"threads_started":1}
I20260806 13:00:35.350474 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.346439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54412 (local address 127.24.83.3:40039)
0806 13:00:35.346606 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:35.346612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:35.346623 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:00:35.346659 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:35.346661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:35.346676 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:35.346769 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:35.346942 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.348014 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.349445 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.350171 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.350237 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.350336 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:35.350365 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 13:00:35.350403 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260806 13:00:35.350458 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.346180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.3:40039 (local address 127.0.0.1:54412)
0806 13:00:35.346454 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:35.346465 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:35.346517 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:35.346791 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:35.346794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:35.346796 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:35.346856 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.346858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.348096 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.348097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.349364 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.349366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.350251 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.350252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.350314 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.350317 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:00:35.350357 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 13:00:35.350388 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":49,"thread_start_us":100,"threads_started":1}
I20260806 13:00:35.351369 13026 tablet_service.cc:1467] Processing CreateTablet for tablet 9adbd1bab46043eda93489de9e0a4cbd (DEFAULT_TABLE table=test-table [id=19123e85a61940eca1b44682b6dabe74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:35.351331 12892 tablet_service.cc:1467] Processing CreateTablet for tablet 9adbd1bab46043eda93489de9e0a4cbd (DEFAULT_TABLE table=test-table [id=19123e85a61940eca1b44682b6dabe74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:35.351730 13026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9adbd1bab46043eda93489de9e0a4cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:35.351732 12892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9adbd1bab46043eda93489de9e0a4cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:35.351933 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.346178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:34671 (local address 127.0.0.1:49458)
0806 13:00:35.346291 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:35.346326 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:35.346389 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:35.347054 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:35.347055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:35.347057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:35.347104 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.347106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.348001 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.348002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.349360 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.349363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.351641 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.351643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.351712 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.351716 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:00:35.351794 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 13:00:35.351841 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":37,"thread_start_us":69,"threads_started":1}
I20260806 13:00:35.351963 13105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.346444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.24.83.1:34671)
0806 13:00:35.346886 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:35.346891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:35.346900 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:00:35.346939 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:35.346941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:35.346957 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:35.347173 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:35.347229 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.347869 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.350590 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.351541 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.351613 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.351694 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:35.351813 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 13:00:35.351854 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":110,"threads_started":1}
I20260806 13:00:35.352567 12757 tablet_service.cc:1467] Processing CreateTablet for tablet 9adbd1bab46043eda93489de9e0a4cbd (DEFAULT_TABLE table=test-table [id=19123e85a61940eca1b44682b6dabe74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:00:35.352869 12757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9adbd1bab46043eda93489de9e0a4cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:35.354581 13107 tablet_bootstrap.cc:492] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258: Bootstrap starting.
I20260806 13:00:35.355085 13108 tablet_bootstrap.cc:492] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: Bootstrap starting.
I20260806 13:00:35.355168 13107 tablet_bootstrap.cc:654] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:35.355513 13107 log.cc:824] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:35.355727 13108 tablet_bootstrap.cc:654] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:35.356112 13108 log.cc:824] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:35.356261 13110 tablet_bootstrap.cc:492] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1: Bootstrap starting.
I20260806 13:00:35.356341 13107 tablet_bootstrap.cc:492] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258: No bootstrap required, opened a new log
I20260806 13:00:35.356408 13107 ts_tablet_manager.cc:1397] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:35.356823 13108 tablet_bootstrap.cc:492] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: No bootstrap required, opened a new log
I20260806 13:00:35.356910 13108 ts_tablet_manager.cc:1397] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:35.356904 13110 tablet_bootstrap.cc:654] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:35.357250 13110 log.cc:824] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:35.358062 13110 tablet_bootstrap.cc:492] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1: No bootstrap required, opened a new log
I20260806 13:00:35.358132 13110 ts_tablet_manager.cc:1397] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:00:35.358987 13108 raft_consensus.cc:348] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3429f5af490d457e8ebe430b1d5c10c1" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34671 } } peers { permanent_uuid: "502bfcad4d1042939aea89179d57d258" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40039 } } peers { permanent_uuid: "21e264971d1045d7a53eae25a0ae6403" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 35669 } }
I20260806 13:00:35.359133 13108 raft_consensus.cc:374] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:35.359179 13108 raft_consensus.cc:729] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21e264971d1045d7a53eae25a0ae6403, State: Initialized, Role: FOLLOWER
I20260806 13:00:35.359084 13107 raft_consensus.cc:348] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3429f5af490d457e8ebe430b1d5c10c1" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34671 } } peers { permanent_uuid: "502bfcad4d1042939aea89179d57d258" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40039 } } peers { permanent_uuid: "21e264971d1045d7a53eae25a0ae6403" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 35669 } }
I20260806 13:00:35.359236 13107 raft_consensus.cc:374] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:35.359289 13107 raft_consensus.cc:729] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 502bfcad4d1042939aea89179d57d258, State: Initialized, Role: FOLLOWER
I20260806 13:00:35.359305 13108 consensus_queue.cc:260] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3429f5af490d457e8ebe430b1d5c10c1" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34671 } } peers { permanent_uuid: "502bfcad4d1042939aea89179d57d258" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40039 } } peers { permanent_uuid: "21e264971d1045d7a53eae25a0ae6403" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 35669 } }
I20260806 13:00:35.359403 13107 consensus_queue.cc:260] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3429f5af490d457e8ebe430b1d5c10c1" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34671 } } peers { permanent_uuid: "502bfcad4d1042939aea89179d57d258" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40039 } } peers { permanent_uuid: "21e264971d1045d7a53eae25a0ae6403" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 35669 } }
I20260806 13:00:35.359609 13108 ts_tablet_manager.cc:1428] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:00:35.359668 13107 ts_tablet_manager.cc:1428] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 13:00:35.359731 12959 heartbeater.cc:503] Master 127.24.83.62:39095 was elected leader, sending a full tablet report...
I20260806 13:00:35.359781 13093 heartbeater.cc:503] Master 127.24.83.62:39095 was elected leader, sending a full tablet report...
I20260806 13:00:35.360674 13110 raft_consensus.cc:348] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3429f5af490d457e8ebe430b1d5c10c1" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34671 } } peers { permanent_uuid: "502bfcad4d1042939aea89179d57d258" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40039 } } peers { permanent_uuid: "21e264971d1045d7a53eae25a0ae6403" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 35669 } }
I20260806 13:00:35.360852 13110 raft_consensus.cc:374] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:35.360927 13110 raft_consensus.cc:729] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3429f5af490d457e8ebe430b1d5c10c1, State: Initialized, Role: FOLLOWER
I20260806 13:00:35.361042 13110 consensus_queue.cc:260] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3429f5af490d457e8ebe430b1d5c10c1" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34671 } } peers { permanent_uuid: "502bfcad4d1042939aea89179d57d258" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40039 } } peers { permanent_uuid: "21e264971d1045d7a53eae25a0ae6403" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 35669 } }
I20260806 13:00:35.361315 13110 ts_tablet_manager.cc:1428] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:00:35.361342 12825 heartbeater.cc:503] Master 127.24.83.62:39095 was elected leader, sending a full tablet report...
W20260806 13:00:35.458752 12960 tablet.cc:2369] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:00:35.481029 13114 raft_consensus.cc:482] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:00:35.481143 13114 raft_consensus.cc:504] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3429f5af490d457e8ebe430b1d5c10c1" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34671 } } peers { permanent_uuid: "502bfcad4d1042939aea89179d57d258" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40039 } } peers { permanent_uuid: "21e264971d1045d7a53eae25a0ae6403" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 35669 } }
I20260806 13:00:35.481513 13114 leader_election.cc:288] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3429f5af490d457e8ebe430b1d5c10c1 (127.24.83.1:34671), 21e264971d1045d7a53eae25a0ae6403 (127.24.83.2:35669)
I20260806 13:00:35.484368 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.481687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:34671 (local address 127.24.83.3:37535)
0806 13:00:35.481743 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:35.481754 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:35.481804 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:35.481975 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:35.481976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:35.481979 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:35.482056 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.482059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.482738 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.482739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.483509 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.483511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.484107 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.484108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.484181 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.484185 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:00:35.484246 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 13:00:35.484292 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":27}
I20260806 13:00:35.484393 13105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.481754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:37535 (local address 127.24.83.1:34671)
0806 13:00:35.481806 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:35.481808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:35.481818 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:00:35.481855 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:35.481856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:35.481869 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:35.481944 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:35.482134 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.482659 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.483605 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.484021 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.484094 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.484102 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:35.484273 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 13:00:35.484318 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:00:35.484632 13106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.481854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.83.3:35049 (local address 127.24.83.2:35669)
0806 13:00:35.481902 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:35.481904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:35.481961 (+ 57us) server_negotiation.cc:407] Connection header received
0806 13:00:35.482013 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:35.482014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:35.482023 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:35.482081 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:00:35.482253 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.482723 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.483645 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.484222 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.484292 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.484301 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:35.484504 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 13:00:35.484556 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:00:35.484728 12779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adbd1bab46043eda93489de9e0a4cbd" candidate_uuid: "502bfcad4d1042939aea89179d57d258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3429f5af490d457e8ebe430b1d5c10c1" is_pre_election: true
I20260806 13:00:35.484867 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.481718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.2:35669 (local address 127.24.83.3:35049)
0806 13:00:35.481889 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:35.481899 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:35.481955 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:35.482108 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:00:35.482109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:00:35.482111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:00:35.482184 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.482186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.482797 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.482798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.483527 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:00:35.483529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.484306 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.484307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:00:35.484391 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.484396 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:00:35.484726 (+ 330us) client_negotiation.cc:240] Negotiation successful
0806 13:00:35.484781 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260806 13:00:35.484915 12779 raft_consensus.cc:2454] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 502bfcad4d1042939aea89179d57d258 in term 0.
I20260806 13:00:35.485191 12980 leader_election.cc:302] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3429f5af490d457e8ebe430b1d5c10c1, 502bfcad4d1042939aea89179d57d258; no voters:
I20260806 13:00:35.485193 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adbd1bab46043eda93489de9e0a4cbd" candidate_uuid: "502bfcad4d1042939aea89179d57d258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e264971d1045d7a53eae25a0ae6403" is_pre_election: true
I20260806 13:00:35.485347 12913 raft_consensus.cc:2454] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 502bfcad4d1042939aea89179d57d258 in term 0.
I20260806 13:00:35.485360 13114 raft_consensus.cc:2781] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:00:35.485442 13114 raft_consensus.cc:482] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:00:35.485481 13114 raft_consensus.cc:3037] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:35.486245 13114 raft_consensus.cc:504] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3429f5af490d457e8ebe430b1d5c10c1" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34671 } } peers { permanent_uuid: "502bfcad4d1042939aea89179d57d258" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40039 } } peers { permanent_uuid: "21e264971d1045d7a53eae25a0ae6403" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 35669 } }
I20260806 13:00:35.486406 13114 leader_election.cc:288] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [CANDIDATE]: Term 1 election: Requested vote from peers 3429f5af490d457e8ebe430b1d5c10c1 (127.24.83.1:34671), 21e264971d1045d7a53eae25a0ae6403 (127.24.83.2:35669)
I20260806 13:00:35.486544 12779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adbd1bab46043eda93489de9e0a4cbd" candidate_uuid: "502bfcad4d1042939aea89179d57d258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3429f5af490d457e8ebe430b1d5c10c1"
I20260806 13:00:35.486560 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adbd1bab46043eda93489de9e0a4cbd" candidate_uuid: "502bfcad4d1042939aea89179d57d258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e264971d1045d7a53eae25a0ae6403"
I20260806 13:00:35.486629 12779 raft_consensus.cc:3037] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:35.486629 12913 raft_consensus.cc:3037] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:35.487530 12913 raft_consensus.cc:2454] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 502bfcad4d1042939aea89179d57d258 in term 1.
I20260806 13:00:35.487530 12779 raft_consensus.cc:2454] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 502bfcad4d1042939aea89179d57d258 in term 1.
I20260806 13:00:35.487705 12978 leader_election.cc:302] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21e264971d1045d7a53eae25a0ae6403, 502bfcad4d1042939aea89179d57d258; no voters:
I20260806 13:00:35.487803 13114 raft_consensus.cc:2781] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:35.488001 13114 raft_consensus.cc:686] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 1 LEADER]: Becoming Leader. State: Replica: 502bfcad4d1042939aea89179d57d258, State: Running, Role: LEADER
I20260806 13:00:35.488124 13114 consensus_queue.cc:237] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3429f5af490d457e8ebe430b1d5c10c1" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34671 } } peers { permanent_uuid: "502bfcad4d1042939aea89179d57d258" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40039 } } peers { permanent_uuid: "21e264971d1045d7a53eae25a0ae6403" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 35669 } }
I20260806 13:00:35.489012 12631 catalog_manager.cc:5495] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 reported cstate change: term changed from 0 to 1, leader changed from <none> to 502bfcad4d1042939aea89179d57d258 (127.24.83.3). New cstate: current_term: 1 leader_uuid: "502bfcad4d1042939aea89179d57d258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3429f5af490d457e8ebe430b1d5c10c1" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "502bfcad4d1042939aea89179d57d258" member_type: VOTER last_known_addr { host: "127.24.83.3" port: 40039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21e264971d1045d7a53eae25a0ae6403" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 35669 } health_report { overall_health: UNKNOWN } } }
I20260806 13:00:35.496325 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:35.494181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54428 (local address 127.24.83.3:40039)
0806 13:00:35.494218 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:35.494220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:35.494244 (+ 24us) server_negotiation.cc:407] Connection header received
0806 13:00:35.494317 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:35.494318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:35.494332 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:35.494388 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:00:35.494485 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.494978 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.495708 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:35.495875 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:35.495970 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:35.496086 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:00:35.496169 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:00:35.496223 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:35.496253 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 13:00:35.496266 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:00:35.497929 12779 raft_consensus.cc:1261] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1 [term 1 FOLLOWER]: Refusing update from remote peer 502bfcad4d1042939aea89179d57d258: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:35.497952 12913 raft_consensus.cc:1261] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403 [term 1 FOLLOWER]: Refusing update from remote peer 502bfcad4d1042939aea89179d57d258: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:00:35.498320 13114 consensus_queue.cc:1035] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21e264971d1045d7a53eae25a0ae6403" member_type: VOTER last_known_addr { host: "127.24.83.2" port: 35669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:35.498476 13117 consensus_queue.cc:1035] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3429f5af490d457e8ebe430b1d5c10c1" member_type: VOTER last_known_addr { host: "127.24.83.1" port: 34671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:00:35.500838 13122 mvcc.cc:205] Tried to move back new op lower bound from 7315542980597157888 to 7315542980560060416. Current Snapshot: MvccSnapshot[applied={T|T < 7315542980597157888}]
I20260806 13:00:35.500916 13120 mvcc.cc:205] Tried to move back new op lower bound from 7315542980597157888 to 7315542980560060416. Current Snapshot: MvccSnapshot[applied={T|T < 7315542980597157888}]
I20260806 13:00:35.503041 12632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:35.503136 12632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:00:35.503829 12632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e563d6951b44991b1d787bd444e5ec8: Sending DeleteTablet for 3 replicas of tablet 9adbd1bab46043eda93489de9e0a4cbd
I20260806 13:00:35.504175 12757 tablet_service.cc:1514] Processing DeleteTablet for tablet 9adbd1bab46043eda93489de9e0a4cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:35 UTC) from {username='slave'} at 127.0.0.1:49458
I20260806 13:00:35.504184 12892 tablet_service.cc:1514] Processing DeleteTablet for tablet 9adbd1bab46043eda93489de9e0a4cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:35 UTC) from {username='slave'} at 127.0.0.1:34422
I20260806 13:00:35.504299 13026 tablet_service.cc:1514] Processing DeleteTablet for tablet 9adbd1bab46043eda93489de9e0a4cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:00:35 UTC) from {username='slave'} at 127.0.0.1:54412
I20260806 13:00:35.504402 13130 tablet_replica.cc:330] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: stopping tablet replica
I20260806 13:00:35.504501 13130 raft_consensus.cc:2229] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:00:35.504611 13130 raft_consensus.cc:2258] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:35.504660 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 12693
I20260806 13:00:35.504782 13131 tablet_replica.cc:330] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1: stopping tablet replica
I20260806 13:00:35.504868 13131 raft_consensus.cc:2229] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:00:35.504946 13131 raft_consensus.cc:2258] T 9adbd1bab46043eda93489de9e0a4cbd P 3429f5af490d457e8ebe430b1d5c10c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:35.505968 13130 ts_tablet_manager.cc:1905] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:00:35.507414 13130 ts_tablet_manager.cc:1918] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:00:35.507503 13130 log.cc:1196] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-1/wal/wals/9adbd1bab46043eda93489de9e0a4cbd
I20260806 13:00:35.507740 13130 ts_tablet_manager.cc:1939] T 9adbd1bab46043eda93489de9e0a4cbd P 21e264971d1045d7a53eae25a0ae6403: Deleting consensus metadata
I20260806 13:00:35.508163 12616 catalog_manager.cc:4841] TS 21e264971d1045d7a53eae25a0ae6403 (127.24.83.2:35669): tablet 9adbd1bab46043eda93489de9e0a4cbd (table test-table [id=19123e85a61940eca1b44682b6dabe74]) successfully deleted
I20260806 13:00:35.512013 13132 tablet_replica.cc:330] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258: stopping tablet replica
I20260806 13:00:35.512128 13132 raft_consensus.cc:2229] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:00:35.512252 13132 raft_consensus.cc:2258] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:00:35.512666 13132 ts_tablet_manager.cc:1905] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:00:35.514019 13132 ts_tablet_manager.cc:1918] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:00:35.514112 13132 log.cc:1196] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258: Deleting WAL directory at /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786020991335182-24908-0/minicluster-data/ts-2/wal/wals/9adbd1bab46043eda93489de9e0a4cbd
W20260806 13:00:35.514240 12617 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.83.1:34671 (error 108)
I20260806 13:00:35.514398 13132 ts_tablet_manager.cc:1939] T 9adbd1bab46043eda93489de9e0a4cbd P 502bfcad4d1042939aea89179d57d258: Deleting consensus metadata
I20260806 13:00:35.514806 12615 catalog_manager.cc:4841] TS 502bfcad4d1042939aea89179d57d258 (127.24.83.3:40039): tablet 9adbd1bab46043eda93489de9e0a4cbd (table test-table [id=19123e85a61940eca1b44682b6dabe74]) successfully deleted
W20260806 13:00:35.514910 13101 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:00:35.514655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.1:34671 (local address 127.0.0.1:49464)
0806 13:00:35.514701 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 13:00:35.514713 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:00:35.514750 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:00:35.514859 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.1:34671: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 13:00:35.515033 12617 catalog_manager.cc:4584] TS 3429f5af490d457e8ebe430b1d5c10c1 (127.24.83.1:34671): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9adbd1bab46043eda93489de9e0a4cbd: Network error: Client connection negotiation failed: client connection to 127.24.83.1:34671: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 13:00:35.515456 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 12828
I20260806 13:00:35.521376 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 12962
I20260806 13:00:35.526753 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 12599
2026-08-06T13:00:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (760 ms)
[----------] 6 tests from EncryptionPolicyTest (8190 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:35 dist-test-slave-zkpd krb5kdc[13136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:35 dist-test-slave-zkpd krb5kdc[13136](info): set up 2 sockets
Aug 06 13:00:35 dist-test-slave-zkpd krb5kdc[13136](info): commencing operation
krb5kdc: starting...
W20260806 13:00:36.566244 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:36 dist-test-slave-zkpd krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:36.680110 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32839
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:36.759471 13151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:36.759694 13151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:36.759732 13151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:36.760870 13151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:36.760938 13151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:36.760963 13151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:36.760989 13151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:36.761020 13151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:36.762449 13151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32839
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:36.762717 13151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:36.762938 13151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:36.767547 13159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:36.767623 13151 server_base.cc:1034] running on GCE node
W20260806 13:00:36.767562 13157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:36.767552 13156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:36.768003 13151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:36.768322 13151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:36.769464 13151 hybrid_clock.cc:648] HybridClock initialized: now 1786021236769438 us; error 39 us; skew 500 ppm
Aug 06 13:00:36 dist-test-slave-zkpd krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:36.772065 13151 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:36.772528 13151 webserver.cc:460] Webserver started at http://127.0.0.1:44237/ using document root <none> and password file <none>
I20260806 13:00:36.772745 13151 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:36.772795 13151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:36.772936 13151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:36.773958 13151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "48c264c2bf774c57b59d63f46afcf6c5"
format_stamp: "Formatted at 2026-08-06 13:00:36 on dist-test-slave-zkpd"
server_key: "8d3030b90ed472504cc6135c0c156a1e"
server_key_iv: "d8718552dec120cc2c385c93375f300f"
server_key_version: "encryptionkey@0"
I20260806 13:00:36.774322 13151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "48c264c2bf774c57b59d63f46afcf6c5"
format_stamp: "Formatted at 2026-08-06 13:00:36 on dist-test-slave-zkpd"
server_key: "8d3030b90ed472504cc6135c0c156a1e"
server_key_iv: "d8718552dec120cc2c385c93375f300f"
server_key_version: "encryptionkey@0"
I20260806 13:00:36.775616 13151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:00:36.776412 13166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:36.776592 13151 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:36.776691 13151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "48c264c2bf774c57b59d63f46afcf6c5"
format_stamp: "Formatted at 2026-08-06 13:00:36 on dist-test-slave-zkpd"
server_key: "8d3030b90ed472504cc6135c0c156a1e"
server_key_iv: "d8718552dec120cc2c385c93375f300f"
server_key_version: "encryptionkey@0"
I20260806 13:00:36.776783 13151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:36.803551 13151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:36.806137 13151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:36.806320 13151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:36.810559 13151 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32839
I20260806 13:00:36.810616 13221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32839 every 8 connection(s)
I20260806 13:00:36.811002 13151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:36.811861 13222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:36.814198 13222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5: Bootstrap starting.
I20260806 13:00:36.814298 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 13151
I20260806 13:00:36.814425 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:36.814621 24908 external_mini_cluster.cc:1428] Setting key a71a1a9324fe587a66ec3976263f4034
I20260806 13:00:36.814889 13222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:36.815244 13222 log.cc:824] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:36.815977 13222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5: No bootstrap required, opened a new log
I20260806 13:00:36.817806 13222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c264c2bf774c57b59d63f46afcf6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32839 } }
I20260806 13:00:36.817973 13222 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:36.818024 13222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c264c2bf774c57b59d63f46afcf6c5, State: Initialized, Role: FOLLOWER
I20260806 13:00:36.818176 13222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c264c2bf774c57b59d63f46afcf6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32839 } }
I20260806 13:00:36.818274 13222 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:36.818318 13222 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:36.818370 13222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:36.818900 13222 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c264c2bf774c57b59d63f46afcf6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32839 } }
I20260806 13:00:36.819020 13222 leader_election.cc:302] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48c264c2bf774c57b59d63f46afcf6c5; no voters:
I20260806 13:00:36.819195 13222 leader_election.cc:288] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:36.819235 13227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:36.819355 13227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [term 1 LEADER]: Becoming Leader. State: Replica: 48c264c2bf774c57b59d63f46afcf6c5, State: Running, Role: LEADER
I20260806 13:00:36.819478 13222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:36.819568 13227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c264c2bf774c57b59d63f46afcf6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32839 } }
I20260806 13:00:36.819973 13227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48c264c2bf774c57b59d63f46afcf6c5. Latest consensus state: current_term: 1 leader_uuid: "48c264c2bf774c57b59d63f46afcf6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c264c2bf774c57b59d63f46afcf6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32839 } } }
I20260806 13:00:36.820119 13227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:36.820168 13229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48c264c2bf774c57b59d63f46afcf6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c264c2bf774c57b59d63f46afcf6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32839 } } }
I20260806 13:00:36.820250 13229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:36.820367 13234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:36.820933 13234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 13:00:36 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 13:00:36.822889 13234 catalog_manager.cc:1347] Generated new cluster ID: 5d8971a6bef24503a55a44589c6f0c9d
I20260806 13:00:36.822947 13234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:36.826033 13225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:36.815141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34599 (local address 127.0.0.1:32839)
0806 13:00:36.815339 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:36.815349 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:36.815382 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:00:36.816035 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:36.816046 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:36.816169 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:36.816343 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:36.816972 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:36.817852 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:36.818776 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:36.819326 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:36.819728 (+ 402us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:36.822248 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:36.822252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:36.822252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:36.822263 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:36.824304 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:36.824753 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:36.824756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:36.824757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:36.824863 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:36.825099 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:36.825100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:36.825100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:36.825425 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:36.825591 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:36.825708 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 13:00:36.825812 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":143,"thread_start_us":82,"threads_started":1}
I20260806 13:00:36.826967 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:36.835681 13225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:36.828514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41321 (local address 127.0.0.1:32839)
0806 13:00:36.828548 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:36.828550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:36.828696 (+ 146us) server_negotiation.cc:407] Connection header received
0806 13:00:36.828724 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:36.828725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:36.828736 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:36.828796 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:36.829374 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:36.830142 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:36.831082 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:36.831416 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:36.831489 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:36.832887 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:36.832889 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:36.832889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:36.832896 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:36.834511 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:36.834903 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:36.834904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:36.834904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:36.834963 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:36.835180 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:36.835180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:36.835181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:36.835308 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:36.835418 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:36.835524 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 13:00:36.835588 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":17}
I20260806 13:00:36.837190 13234 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:36.837707 13234 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:36.846101 13234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48c264c2bf774c57b59d63f46afcf6c5: Generated new TSK 0
I20260806 13:00:36.846297 13234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:36.847857 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 13151
W20260806 13:00:36.852916 13134 connection.cc:535] client connection to 127.0.0.1:32839 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32839): Connection reset by peer (error 104)
2026-08-06T13:00:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:36 dist-test-slave-zkpd krb5kdc[13254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:36 dist-test-slave-zkpd krb5kdc[13254](info): set up 2 sockets
Aug 06 13:00:36 dist-test-slave-zkpd krb5kdc[13254](info): commencing operation
krb5kdc: starting...
W20260806 13:00:37.884265 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:37 dist-test-slave-zkpd krb5kdc[13254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:37.996809 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42599
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:38.078325 13267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:38.078518 13267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:38.078541 13267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:38.079679 13267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:38.079720 13267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:38.079734 13267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:38.079777 13267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:38.079803 13267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:38.081321 13267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42599
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:38.081555 13267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:38.081775 13267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:38.086589 13272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:38.086597 13275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:38.086787 13273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:38.086802 13267 server_base.cc:1034] running on GCE node
I20260806 13:00:38.087036 13267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:38.087358 13267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:38.088500 13267 hybrid_clock.cc:648] HybridClock initialized: now 1786021238088489 us; error 30 us; skew 500 ppm
Aug 06 13:00:38 dist-test-slave-zkpd krb5kdc[13254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:38.095484 13267 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:38.096006 13267 webserver.cc:460] Webserver started at http://127.0.0.1:33701/ using document root <none> and password file <none>
I20260806 13:00:38.096230 13267 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:38.096276 13267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:38.096424 13267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:38.097468 13267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "ef3e2c264c8b4c09a2dfda8097524027"
format_stamp: "Formatted at 2026-08-06 13:00:38 on dist-test-slave-zkpd"
server_key: "20af18419a13d0b8536d98b14d895942"
server_key_iv: "803af98b608cdf3e8589733cda6872e8"
server_key_version: "encryptionkey@0"
I20260806 13:00:38.097831 13267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "ef3e2c264c8b4c09a2dfda8097524027"
format_stamp: "Formatted at 2026-08-06 13:00:38 on dist-test-slave-zkpd"
server_key: "20af18419a13d0b8536d98b14d895942"
server_key_iv: "803af98b608cdf3e8589733cda6872e8"
server_key_version: "encryptionkey@0"
I20260806 13:00:38.099123 13267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:00:38.099813 13282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:38.099997 13267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:38.100095 13267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "ef3e2c264c8b4c09a2dfda8097524027"
format_stamp: "Formatted at 2026-08-06 13:00:38 on dist-test-slave-zkpd"
server_key: "20af18419a13d0b8536d98b14d895942"
server_key_iv: "803af98b608cdf3e8589733cda6872e8"
server_key_version: "encryptionkey@0"
I20260806 13:00:38.100174 13267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:38.126179 13267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:38.128842 13267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:38.129017 13267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:38.133314 13267 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42599
I20260806 13:00:38.133347 13336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42599 every 8 connection(s)
I20260806 13:00:38.133714 13267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:38.134562 13337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:38.136843 13337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027: Bootstrap starting.
I20260806 13:00:38.137434 13337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:38.137720 13337 log.cc:824] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:38.138434 13337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027: No bootstrap required, opened a new log
I20260806 13:00:38.140029 13337 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef3e2c264c8b4c09a2dfda8097524027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } }
I20260806 13:00:38.140162 13337 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:38.140184 13337 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef3e2c264c8b4c09a2dfda8097524027, State: Initialized, Role: FOLLOWER
I20260806 13:00:38.140341 13337 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef3e2c264c8b4c09a2dfda8097524027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } }
I20260806 13:00:38.140403 13337 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:38.140446 13337 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:38.140496 13337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:38.141031 13337 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef3e2c264c8b4c09a2dfda8097524027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } }
I20260806 13:00:38.141119 13337 leader_election.cc:302] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef3e2c264c8b4c09a2dfda8097524027; no voters:
I20260806 13:00:38.141305 13337 leader_election.cc:288] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:38.141371 13340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:38.141544 13337 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:38.141613 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 13267
I20260806 13:00:38.141637 13340 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [term 1 LEADER]: Becoming Leader. State: Replica: ef3e2c264c8b4c09a2dfda8097524027, State: Running, Role: LEADER
I20260806 13:00:38.141742 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:38.141731 13340 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef3e2c264c8b4c09a2dfda8097524027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } }
I20260806 13:00:38.141968 24908 external_mini_cluster.cc:1428] Setting key 0a85326bb039fa927947b29b67a37368
I20260806 13:00:38.142082 13340 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef3e2c264c8b4c09a2dfda8097524027. Latest consensus state: current_term: 1 leader_uuid: "ef3e2c264c8b4c09a2dfda8097524027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef3e2c264c8b4c09a2dfda8097524027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } } }
I20260806 13:00:38.142163 13340 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:38.142342 13341 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef3e2c264c8b4c09a2dfda8097524027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef3e2c264c8b4c09a2dfda8097524027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } } }
I20260806 13:00:38.142432 13341 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027 [sys.catalog]: This master's current role is: LEADER
W20260806 13:00:38.143553 13357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:00:38.143608 13357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:00:38.143651 13349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:38.143817 13349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:38.145331 13349 catalog_manager.cc:1347] Generated new cluster ID: 93ac0d2823d048c3b1648103cd6868cd
I20260806 13:00:38.145385 13349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:38.148052 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:38.142642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59035 (local address 127.0.0.1:42599)
0806 13:00:38.142823 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:38.142833 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:38.142866 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:00:38.142985 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:38.142993 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:38.143053 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:38.143452 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:38.143583 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:38.144821 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:38.145928 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:38.146509 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:38.146608 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:38.147359 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:38.147363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:38.147363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:38.147569 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:38.147608 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:38.147622 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:38.147696 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:38.147805 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 13:00:38.147829 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260806 13:00:38.149250 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:38.154439 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:38.150833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39873 (local address 127.0.0.1:42599)
0806 13:00:38.150865 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:38.150867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:38.151066 (+ 199us) server_negotiation.cc:407] Connection header received
0806 13:00:38.151142 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:38.151143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:38.151152 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:38.151178 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:38.151347 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:38.152075 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:38.153110 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:38.153410 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:38.153468 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:38.154192 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:38.154193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:38.154194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:38.154198 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:38.154211 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:38.154215 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:38.154273 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:38.154356 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 13:00:38.154367 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:00:38.158201 13349 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:38.158886 13349 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:38.163621 13349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef3e2c264c8b4c09a2dfda8097524027: Generated new TSK 0
I20260806 13:00:38.163829 13349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:38.166849 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 13267
W20260806 13:00:38.172854 13252 connection.cc:535] client connection to 127.0.0.1:42599 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42599): Connection reset by peer (error 104)
2026-08-06T13:00:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:38 dist-test-slave-zkpd krb5kdc[13370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:38 dist-test-slave-zkpd krb5kdc[13370](info): set up 2 sockets
Aug 06 13:00:38 dist-test-slave-zkpd krb5kdc[13370](info): commencing operation
krb5kdc: starting...
W20260806 13:00:39.244096 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:39 dist-test-slave-zkpd krb5kdc[13370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:39.359632 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40857
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:39.439316 13383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:39.439502 13383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:39.439523 13383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:39.440618 13383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:39.440650 13383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:39.440665 13383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:39.440676 13383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:39.440688 13383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:39.442065 13383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40857
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:39.442245 13383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:39.442413 13383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 13:00:39.447359 13383 server_base.cc:1034] running on GCE node
W20260806 13:00:39.447291 13389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:39.447295 13392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:39.447336 13390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:39.447741 13383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:39.448055 13383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:39.449199 13383 hybrid_clock.cc:648] HybridClock initialized: now 1786021239449185 us; error 28 us; skew 500 ppm
Aug 06 13:00:39 dist-test-slave-zkpd krb5kdc[13370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:39.451563 13383 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:39.452114 13383 webserver.cc:460] Webserver started at http://127.0.0.1:40625/ using document root <none> and password file <none>
I20260806 13:00:39.452333 13383 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:39.452378 13383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:39.452520 13383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:39.453557 13383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "3db136559ce54f9cbc30f5f25840798a"
format_stamp: "Formatted at 2026-08-06 13:00:39 on dist-test-slave-zkpd"
server_key: "9c0ddb2baa0c54294745eab81f0f4a00"
server_key_iv: "894ed472890f82f68b608a14534ad218"
server_key_version: "encryptionkey@0"
I20260806 13:00:39.453924 13383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "3db136559ce54f9cbc30f5f25840798a"
format_stamp: "Formatted at 2026-08-06 13:00:39 on dist-test-slave-zkpd"
server_key: "9c0ddb2baa0c54294745eab81f0f4a00"
server_key_iv: "894ed472890f82f68b608a14534ad218"
server_key_version: "encryptionkey@0"
I20260806 13:00:39.455197 13383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:00:39.455881 13399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:39.456096 13383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:39.456182 13383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "3db136559ce54f9cbc30f5f25840798a"
format_stamp: "Formatted at 2026-08-06 13:00:39 on dist-test-slave-zkpd"
server_key: "9c0ddb2baa0c54294745eab81f0f4a00"
server_key_iv: "894ed472890f82f68b608a14534ad218"
server_key_version: "encryptionkey@0"
I20260806 13:00:39.456252 13383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:39.460425 13383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:39.463160 13383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:39.463342 13383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:39.468220 13383 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40857
I20260806 13:00:39.468600 13383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:39.472040 13452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40857 every 8 connection(s)
I20260806 13:00:39.473026 13453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:39.473577 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 13383
I20260806 13:00:39.473672 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:39.473819 24908 external_mini_cluster.cc:1428] Setting key b627f10180267e036d6fc0923525602a
I20260806 13:00:39.476228 13453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a: Bootstrap starting.
I20260806 13:00:39.476395 13456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:39.474297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35393 (local address 127.0.0.1:40857)
0806 13:00:39.474514 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:39.474524 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:39.474558 (+ 34us) server_negotiation.cc:407] Connection header received
0806 13:00:39.475095 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:39.475103 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:39.475136 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:39.475285 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:39.475921 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:39.475924 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:39.475925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:39.475929 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:39.475974 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:39.475989 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:39.476070 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:39.476129 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 13:00:39.476168 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":97,"threads_started":1}
I20260806 13:00:39.476943 13453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:39.477284 13453 log.cc:824] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:39.477394 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:39.477949 13453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a: No bootstrap required, opened a new log
I20260806 13:00:39.479872 13456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:39.478877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52043 (local address 127.0.0.1:40857)
0806 13:00:39.478926 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:39.478928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:39.479015 (+ 87us) server_negotiation.cc:407] Connection header received
0806 13:00:39.479076 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:39.479077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:39.479082 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:39.479142 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:39.479638 (+ 496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:39.479639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:39.479639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:39.479644 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:39.479662 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:39.479666 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:39.479740 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:39.479780 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 13:00:39.479794 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 13:00:39.480291 13453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db136559ce54f9cbc30f5f25840798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40857 } }
I20260806 13:00:39.480456 13453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:39.480482 13453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3db136559ce54f9cbc30f5f25840798a, State: Initialized, Role: FOLLOWER
I20260806 13:00:39.480567 13453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db136559ce54f9cbc30f5f25840798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40857 } }
I20260806 13:00:39.480630 13453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:39.480651 13453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:39.480677 13453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:39.481168 13453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db136559ce54f9cbc30f5f25840798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40857 } }
I20260806 13:00:39.481251 13453 leader_election.cc:302] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3db136559ce54f9cbc30f5f25840798a; no voters:
I20260806 13:00:39.481372 13453 leader_election.cc:288] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:39.481434 13464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:39.481551 13453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:39.481590 13464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [term 1 LEADER]: Becoming Leader. State: Replica: 3db136559ce54f9cbc30f5f25840798a, State: Running, Role: LEADER
I20260806 13:00:39.481681 13464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db136559ce54f9cbc30f5f25840798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40857 } }
I20260806 13:00:39.481986 13464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3db136559ce54f9cbc30f5f25840798a. Latest consensus state: current_term: 1 leader_uuid: "3db136559ce54f9cbc30f5f25840798a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db136559ce54f9cbc30f5f25840798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40857 } } }
I20260806 13:00:39.481995 13465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3db136559ce54f9cbc30f5f25840798a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3db136559ce54f9cbc30f5f25840798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40857 } } }
I20260806 13:00:39.482077 13464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:39.482084 13465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:39.482314 13470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:39.482950 13470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:39.484483 13470 catalog_manager.cc:1347] Generated new cluster ID: 534d1175868c46f1a7e7a305f5ede58b
I20260806 13:00:39.484535 13470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:39.492290 13470 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:39.492444 13470 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:39.508940 13470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3db136559ce54f9cbc30f5f25840798a: Generated new TSK 0
I20260806 13:00:39.509140 13470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:39.515715 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 13383
2026-08-06T13:00:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:39 dist-test-slave-zkpd krb5kdc[13485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:39 dist-test-slave-zkpd krb5kdc[13485](info): set up 2 sockets
Aug 06 13:00:39 dist-test-slave-zkpd krb5kdc[13485](info): commencing operation
krb5kdc: starting...
W20260806 13:00:40.552114 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:40 dist-test-slave-zkpd krb5kdc[13485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:40.668735 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32841
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:40.748348 13498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:40.748548 13498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:40.748570 13498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:40.749719 13498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:40.749760 13498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:40.749774 13498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:40.749820 13498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:40.749850 13498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:40.751327 13498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:40.751551 13498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:40.751777 13498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:40.756417 13506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:40.756425 13503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:40.756417 13504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:40.756637 13498 server_base.cc:1034] running on GCE node
I20260806 13:00:40.756865 13498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:40.757186 13498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:40.758334 13498 hybrid_clock.cc:648] HybridClock initialized: now 1786021240758322 us; error 42 us; skew 500 ppm
Aug 06 13:00:40 dist-test-slave-zkpd krb5kdc[13485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021240, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:40.760744 13498 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:40.761158 13498 webserver.cc:460] Webserver started at http://127.0.0.1:37577/ using document root <none> and password file <none>
I20260806 13:00:40.761394 13498 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:40.761462 13498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:40.761615 13498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:40.762660 13498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "9fb868a419464633abfcca053d47db74"
format_stamp: "Formatted at 2026-08-06 13:00:40 on dist-test-slave-zkpd"
server_key: "5563459c05168e6e2e5c532c04c18776"
server_key_iv: "08c790b26080a37a160748c6eb0020a0"
server_key_version: "encryptionkey@0"
I20260806 13:00:40.763039 13498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "9fb868a419464633abfcca053d47db74"
format_stamp: "Formatted at 2026-08-06 13:00:40 on dist-test-slave-zkpd"
server_key: "5563459c05168e6e2e5c532c04c18776"
server_key_iv: "08c790b26080a37a160748c6eb0020a0"
server_key_version: "encryptionkey@0"
I20260806 13:00:40.764485 13498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:00:40.765244 13514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:40.765421 13498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:40.765528 13498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "9fb868a419464633abfcca053d47db74"
format_stamp: "Formatted at 2026-08-06 13:00:40 on dist-test-slave-zkpd"
server_key: "5563459c05168e6e2e5c532c04c18776"
server_key_iv: "08c790b26080a37a160748c6eb0020a0"
server_key_version: "encryptionkey@0"
I20260806 13:00:40.765602 13498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:40.778229 13498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:40.780200 13498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:40.780368 13498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:40.784431 13498 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32841
I20260806 13:00:40.784466 13567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32841 every 8 connection(s)
I20260806 13:00:40.784821 13498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:40.785665 13568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:40.787973 13568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74: Bootstrap starting.
I20260806 13:00:40.788532 13568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:40.788822 13568 log.cc:824] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:40.789512 13568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74: No bootstrap required, opened a new log
I20260806 13:00:40.791087 13568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb868a419464633abfcca053d47db74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32841 } }
I20260806 13:00:40.791227 13568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:40.791250 13568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fb868a419464633abfcca053d47db74, State: Initialized, Role: FOLLOWER
I20260806 13:00:40.791409 13568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb868a419464633abfcca053d47db74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32841 } }
I20260806 13:00:40.791471 13568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:40.791492 13568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:40.791558 13568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:40.792179 13568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb868a419464633abfcca053d47db74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32841 } }
I20260806 13:00:40.792291 13568 leader_election.cc:302] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fb868a419464633abfcca053d47db74; no voters:
I20260806 13:00:40.792466 13568 leader_election.cc:288] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:40.792534 13571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:40.792729 13568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:40.792713 13571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [term 1 LEADER]: Becoming Leader. State: Replica: 9fb868a419464633abfcca053d47db74, State: Running, Role: LEADER
I20260806 13:00:40.792866 13571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb868a419464633abfcca053d47db74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32841 } }
I20260806 13:00:40.793223 13573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fb868a419464633abfcca053d47db74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb868a419464633abfcca053d47db74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32841 } } }
I20260806 13:00:40.793334 13573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:40.793324 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 13498
I20260806 13:00:40.793226 13574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fb868a419464633abfcca053d47db74. Latest consensus state: current_term: 1 leader_uuid: "9fb868a419464633abfcca053d47db74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb868a419464633abfcca053d47db74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32841 } } }
I20260806 13:00:40.793430 13574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:40.793453 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:40.793565 13576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:40.793641 24908 external_mini_cluster.cc:1428] Setting key 7f496fb62f3ca444047679062eebad5c
I20260806 13:00:40.794278 13576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:40.796252 13576 catalog_manager.cc:1347] Generated new cluster ID: e6e54a669b7b4bf2a5fdc76899cad74f
I20260806 13:00:40.796307 13576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:00:40 dist-test-slave-zkpd krb5kdc[13485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 13:00:40.806474 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:40.794332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57973 (local address 127.0.0.1:32841)
0806 13:00:40.795298 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:40.795308 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:40.795339 (+ 31us) server_negotiation.cc:407] Connection header received
0806 13:00:40.795560 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:40.795566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:40.795618 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:40.795774 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:40.796585 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:40.798031 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:40.799084 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:40.799701 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:40.799828 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:40.802620 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:40.802624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:40.802625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:40.802638 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:40.804657 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:40.805080 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:40.805083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:40.805084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:40.805201 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:40.805414 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:40.805415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:40.805415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:40.805709 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:40.805855 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:40.805987 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 13:00:40.806246 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":83,"threads_started":1}
I20260806 13:00:40.807412 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:40.813341 13576 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:40.813917 13576 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:40.819782 13576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fb868a419464633abfcca053d47db74: Generated new TSK 0
I20260806 13:00:40.819979 13576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:40.820556 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:40.813016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42113 (local address 127.0.0.1:32841)
0806 13:00:40.813070 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:40.813072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:40.813320 (+ 248us) server_negotiation.cc:407] Connection header received
0806 13:00:40.813404 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:40.813405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:40.813418 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:40.813479 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:40.814153 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:40.814856 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:40.815871 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:40.816198 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:40.816257 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:40.817700 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:40.817702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:40.817702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:40.817709 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:40.819297 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:40.819679 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:40.819680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:40.819681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:40.819741 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:40.820020 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:40.820021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:40.820021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:40.820151 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:40.820250 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:40.820391 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 13:00:40.820476 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:00:40.822289 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 13498
2026-08-06T13:00:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:40 dist-test-slave-zkpd krb5kdc[13600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:40 dist-test-slave-zkpd krb5kdc[13600](info): set up 2 sockets
Aug 06 13:00:40 dist-test-slave-zkpd krb5kdc[13600](info): commencing operation
krb5kdc: starting...
W20260806 13:00:41.872355 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:41 dist-test-slave-zkpd krb5kdc[13600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:41.986863 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36381
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:42.066309 13613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:42.066542 13613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:42.066586 13613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:42.067725 13613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:42.067799 13613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:42.067823 13613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:42.067852 13613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:42.067914 13613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:42.069401 13613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:42.069669 13613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:42.069905 13613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:42.074707 13618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:42.074719 13619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:42.074683 13621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:42.074935 13613 server_base.cc:1034] running on GCE node
I20260806 13:00:42.075191 13613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:42.075495 13613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:42.076633 13613 hybrid_clock.cc:648] HybridClock initialized: now 1786021242076615 us; error 38 us; skew 500 ppm
Aug 06 13:00:42 dist-test-slave-zkpd krb5kdc[13600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021242, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:42.078948 13613 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:42.079351 13613 webserver.cc:460] Webserver started at http://127.0.0.1:44863/ using document root <none> and password file <none>
I20260806 13:00:42.079587 13613 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:42.079654 13613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:42.079804 13613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:42.080852 13613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "2600a613fc6b4b619ec8a2a3bde7b6f4"
format_stamp: "Formatted at 2026-08-06 13:00:42 on dist-test-slave-zkpd"
server_key: "0686c459d6ce8cf411815ec38e74d1a6"
server_key_iv: "4923f8c86f6d71de677bb95e198f58c3"
server_key_version: "encryptionkey@0"
I20260806 13:00:42.081208 13613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "2600a613fc6b4b619ec8a2a3bde7b6f4"
format_stamp: "Formatted at 2026-08-06 13:00:42 on dist-test-slave-zkpd"
server_key: "0686c459d6ce8cf411815ec38e74d1a6"
server_key_iv: "4923f8c86f6d71de677bb95e198f58c3"
server_key_version: "encryptionkey@0"
I20260806 13:00:42.082513 13613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:00:42.083245 13628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:42.083421 13613 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:00:42.083519 13613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "2600a613fc6b4b619ec8a2a3bde7b6f4"
format_stamp: "Formatted at 2026-08-06 13:00:42 on dist-test-slave-zkpd"
server_key: "0686c459d6ce8cf411815ec38e74d1a6"
server_key_iv: "4923f8c86f6d71de677bb95e198f58c3"
server_key_version: "encryptionkey@0"
I20260806 13:00:42.083609 13613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:42.108548 13613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:42.110348 13613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:42.110533 13613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:42.114620 13613 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36381
I20260806 13:00:42.114658 13682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36381 every 8 connection(s)
I20260806 13:00:42.115026 13613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:42.115864 13683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:42.118096 13683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4: Bootstrap starting.
I20260806 13:00:42.118599 13683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:42.118837 13683 log.cc:824] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:42.119478 13683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4: No bootstrap required, opened a new log
I20260806 13:00:42.121021 13683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2600a613fc6b4b619ec8a2a3bde7b6f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } }
I20260806 13:00:42.121158 13683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:42.121181 13683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2600a613fc6b4b619ec8a2a3bde7b6f4, State: Initialized, Role: FOLLOWER
I20260806 13:00:42.121279 13683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2600a613fc6b4b619ec8a2a3bde7b6f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } }
I20260806 13:00:42.121328 13683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:42.121349 13683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:42.121379 13683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:42.121620 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 13613
I20260806 13:00:42.121739 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:42.121927 24908 external_mini_cluster.cc:1428] Setting key 2cacee73fce4a6de3bab74e9a45efb8c
I20260806 13:00:42.122104 13683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2600a613fc6b4b619ec8a2a3bde7b6f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } }
I20260806 13:00:42.122234 13683 leader_election.cc:302] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2600a613fc6b4b619ec8a2a3bde7b6f4; no voters:
I20260806 13:00:42.122392 13683 leader_election.cc:288] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:42.122565 13690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:42.122653 13683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:42.122678 13690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [term 1 LEADER]: Becoming Leader. State: Replica: 2600a613fc6b4b619ec8a2a3bde7b6f4, State: Running, Role: LEADER
I20260806 13:00:42.122753 13690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2600a613fc6b4b619ec8a2a3bde7b6f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } }
I20260806 13:00:42.123087 13690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2600a613fc6b4b619ec8a2a3bde7b6f4. Latest consensus state: current_term: 1 leader_uuid: "2600a613fc6b4b619ec8a2a3bde7b6f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2600a613fc6b4b619ec8a2a3bde7b6f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } } }
I20260806 13:00:42.123169 13690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:42.123098 13691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2600a613fc6b4b619ec8a2a3bde7b6f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2600a613fc6b4b619ec8a2a3bde7b6f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } } }
I20260806 13:00:42.123294 13691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:42.123857 13693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:42.124418 13693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:42.126034 13693 catalog_manager.cc:1347] Generated new cluster ID: 008999dc9f73478fab7970fe00a23b38
I20260806 13:00:42.126083 13693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:42.128782 13689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:42.122358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51629 (local address 127.0.0.1:36381)
0806 13:00:42.123378 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:42.123390 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:42.123427 (+ 37us) server_negotiation.cc:407] Connection header received
0806 13:00:42.123561 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:42.123569 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:42.123634 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:42.123829 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:42.124129 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:42.125427 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:42.126514 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:42.127098 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:42.127201 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:42.127943 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:42.127947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:42.127947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:42.128178 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:42.128213 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:42.128227 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:42.128282 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:42.128518 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 13:00:42.128577 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":967,"thread_start_us":63,"threads_started":1}
I20260806 13:00:42.129701 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:42.134416 13689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:42.131078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34245 (local address 127.0.0.1:36381)
0806 13:00:42.131119 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:42.131120 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:42.131225 (+ 105us) server_negotiation.cc:407] Connection header received
0806 13:00:42.131269 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:42.131269 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:42.131281 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:42.131349 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:42.131447 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:42.132193 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:42.133157 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:42.133459 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:42.133533 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:42.134178 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:42.134179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:42.134179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:42.134184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:42.134198 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:42.134202 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:42.134261 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:42.134320 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 13:00:42.134344 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:00:42.145802 13693 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:42.146539 13693 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:42.171111 13693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2600a613fc6b4b619ec8a2a3bde7b6f4: Generated new TSK 0
I20260806 13:00:42.171360 13693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:42.179150 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 13613
2026-08-06T13:00:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:42 dist-test-slave-zkpd krb5kdc[13715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:42 dist-test-slave-zkpd krb5kdc[13715](info): set up 2 sockets
Aug 06 13:00:42 dist-test-slave-zkpd krb5kdc[13715](info): commencing operation
krb5kdc: starting...
W20260806 13:00:43.216518 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:43 dist-test-slave-zkpd krb5kdc[13715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:43.335202 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:43.419783 13728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:43.420068 13728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:43.420097 13728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:43.421207 13728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:43.421242 13728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:43.421257 13728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:43.421269 13728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:43.421281 13728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:43.422657 13728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:43.422863 13728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:43.423036 13728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:43.427734 13736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:43.427742 13734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:43.427740 13733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:43.428011 13728 server_base.cc:1034] running on GCE node
I20260806 13:00:43.428303 13728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:43.428624 13728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:43.429791 13728 hybrid_clock.cc:648] HybridClock initialized: now 1786021243429771 us; error 33 us; skew 500 ppm
Aug 06 13:00:43 dist-test-slave-zkpd krb5kdc[13715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:43.437218 13728 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:43.437615 13728 webserver.cc:460] Webserver started at http://127.0.0.1:34405/ using document root <none> and password file <none>
I20260806 13:00:43.437794 13728 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:43.437834 13728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:43.437932 13728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:43.438938 13728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "381611a483d04058b87ad6be870749ed"
format_stamp: "Formatted at 2026-08-06 13:00:43 on dist-test-slave-zkpd"
server_key: "fa72771307bcd80b07162cd02fdd717d"
server_key_iv: "6ff9125a4ccab211eda17da89ceaa2ca"
server_key_version: "encryptionkey@0"
I20260806 13:00:43.439234 13728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "381611a483d04058b87ad6be870749ed"
format_stamp: "Formatted at 2026-08-06 13:00:43 on dist-test-slave-zkpd"
server_key: "fa72771307bcd80b07162cd02fdd717d"
server_key_iv: "6ff9125a4ccab211eda17da89ceaa2ca"
server_key_version: "encryptionkey@0"
I20260806 13:00:43.440563 13728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:00:43.441242 13743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:43.441431 13728 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:43.441505 13728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "381611a483d04058b87ad6be870749ed"
format_stamp: "Formatted at 2026-08-06 13:00:43 on dist-test-slave-zkpd"
server_key: "fa72771307bcd80b07162cd02fdd717d"
server_key_iv: "6ff9125a4ccab211eda17da89ceaa2ca"
server_key_version: "encryptionkey@0"
I20260806 13:00:43.441558 13728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:43.444720 13728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:43.446955 13728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:43.447072 13728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:43.451104 13728 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45335
I20260806 13:00:43.451136 13798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45335 every 8 connection(s)
I20260806 13:00:43.451457 13728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:43.452529 13799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:43.454815 13799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed: Bootstrap starting.
I20260806 13:00:43.455387 13799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:43.455668 13799 log.cc:824] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:43.456413 13799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed: No bootstrap required, opened a new log
I20260806 13:00:43.457989 13799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "381611a483d04058b87ad6be870749ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45335 } }
I20260806 13:00:43.458182 13799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:43.458218 13799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 381611a483d04058b87ad6be870749ed, State: Initialized, Role: FOLLOWER
I20260806 13:00:43.458364 13799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "381611a483d04058b87ad6be870749ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45335 } }
I20260806 13:00:43.458451 13799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:43.458487 13799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:43.458540 13799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:43.459300 13799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "381611a483d04058b87ad6be870749ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45335 } }
I20260806 13:00:43.459441 13799 leader_election.cc:302] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 381611a483d04058b87ad6be870749ed; no voters:
I20260806 13:00:43.459614 13799 leader_election.cc:288] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:43.459668 13802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:43.459820 13802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [term 1 LEADER]: Becoming Leader. State: Replica: 381611a483d04058b87ad6be870749ed, State: Running, Role: LEADER
I20260806 13:00:43.459914 13799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:43.459940 13802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "381611a483d04058b87ad6be870749ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45335 } }
I20260806 13:00:43.460335 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 13728
I20260806 13:00:43.460453 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:43.460652 24908 external_mini_cluster.cc:1428] Setting key d0585d392d96f2212d3c06fa05f75b57
I20260806 13:00:43.461153 13804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "381611a483d04058b87ad6be870749ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "381611a483d04058b87ad6be870749ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45335 } } }
I20260806 13:00:43.461257 13804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:43.461402 13804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 381611a483d04058b87ad6be870749ed. Latest consensus state: current_term: 1 leader_uuid: "381611a483d04058b87ad6be870749ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "381611a483d04058b87ad6be870749ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45335 } } }
I20260806 13:00:43.461468 13804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:43.461617 13814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:43.462206 13814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:43.466027 13821 mvcc.cc:205] Tried to move back new op lower bound from 7315543013225705472 to 7315543013212422144. Current Snapshot: MvccSnapshot[applied={T|T < 7315543013225705472 or (T in {7315543013225705472})}]
I20260806 13:00:43.466202 13814 catalog_manager.cc:1347] Generated new cluster ID: ff97e71fe8f548908970efd03708f370
I20260806 13:00:43.466249 13814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:43.466794 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:43.461147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51323 (local address 127.0.0.1:45335)
0806 13:00:43.463108 (+ 1961us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:43.463117 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:43.464477 (+ 1360us) server_negotiation.cc:407] Connection header received
0806 13:00:43.464661 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:43.464668 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:43.464738 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:43.464944 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:43.466246 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:43.466250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:43.466250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:43.466254 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:43.466296 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:43.466311 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:43.466367 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:43.466537 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 13:00:43.466583 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1899,"thread_start_us":75,"threads_started":1}
I20260806 13:00:43.468317 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:43.472927 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:43.469851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45499 (local address 127.0.0.1:45335)
0806 13:00:43.471973 (+ 2122us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:43.471975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:43.471983 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:00:43.472004 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:43.472004 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:43.472009 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:43.472064 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:43.472689 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:43.472690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:43.472691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:43.472695 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:43.472708 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:43.472712 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:43.472763 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:43.472846 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 13:00:43.472858 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2103}
I20260806 13:00:43.480450 13814 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:43.480623 13814 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:43.486920 13814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 381611a483d04058b87ad6be870749ed: Generated new TSK 0
I20260806 13:00:43.487093 13814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:43.489511 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 13728
2026-08-06T13:00:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:43 dist-test-slave-zkpd krb5kdc[13831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:43 dist-test-slave-zkpd krb5kdc[13831](info): set up 2 sockets
Aug 06 13:00:43 dist-test-slave-zkpd krb5kdc[13831](info): commencing operation
krb5kdc: starting...
W20260806 13:00:44.524277 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:44 dist-test-slave-zkpd krb5kdc[13831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:44.638948 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34985
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:44.718370 13845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:44.718601 13845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:44.718644 13845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:44.719777 13845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:44.719866 13845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:44.719888 13845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:44.719913 13845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:44.719964 13845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:44.721419 13845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:44.721709 13845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:44.721931 13845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:44.726719 13851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:44.726730 13850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:44.726845 13845 server_base.cc:1034] running on GCE node
W20260806 13:00:44.726728 13853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:44.727200 13845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:44.727474 13845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:44.728611 13845 hybrid_clock.cc:648] HybridClock initialized: now 1786021244728599 us; error 32 us; skew 500 ppm
Aug 06 13:00:44 dist-test-slave-zkpd krb5kdc[13831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021244, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:44.730953 13845 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:44.731317 13845 webserver.cc:460] Webserver started at http://127.0.0.1:32835/ using document root <none> and password file <none>
I20260806 13:00:44.731499 13845 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:44.731541 13845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:44.731638 13845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:44.732676 13845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "fb97270c4f16485bb748d17b51458ece"
format_stamp: "Formatted at 2026-08-06 13:00:44 on dist-test-slave-zkpd"
server_key: "ceac4100a883902ee450c813ef6d3f89"
server_key_iv: "a0b82cbe2518ded70a857f8d980c9713"
server_key_version: "encryptionkey@0"
I20260806 13:00:44.732988 13845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "fb97270c4f16485bb748d17b51458ece"
format_stamp: "Formatted at 2026-08-06 13:00:44 on dist-test-slave-zkpd"
server_key: "ceac4100a883902ee450c813ef6d3f89"
server_key_iv: "a0b82cbe2518ded70a857f8d980c9713"
server_key_version: "encryptionkey@0"
I20260806 13:00:44.734279 13845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:00:44.734948 13860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:44.735137 13845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:00:44.735242 13845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "fb97270c4f16485bb748d17b51458ece"
format_stamp: "Formatted at 2026-08-06 13:00:44 on dist-test-slave-zkpd"
server_key: "ceac4100a883902ee450c813ef6d3f89"
server_key_iv: "a0b82cbe2518ded70a857f8d980c9713"
server_key_version: "encryptionkey@0"
I20260806 13:00:44.735317 13845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:44.751174 13845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:44.753014 13845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:44.753163 13845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:44.757085 13845 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34985
I20260806 13:00:44.757164 13914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34985 every 8 connection(s)
I20260806 13:00:44.757488 13845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:44.758343 13915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:44.760635 13915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece: Bootstrap starting.
I20260806 13:00:44.761206 13915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:44.761502 13915 log.cc:824] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:44.762182 13915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece: No bootstrap required, opened a new log
I20260806 13:00:44.763411 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 13845
I20260806 13:00:44.763517 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:44.763729 24908 external_mini_cluster.cc:1428] Setting key e4866b2a82a9ba04ce7ae239c54715a3
I20260806 13:00:44.764034 13915 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb97270c4f16485bb748d17b51458ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34985 } }
I20260806 13:00:44.764201 13915 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:44.764245 13915 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb97270c4f16485bb748d17b51458ece, State: Initialized, Role: FOLLOWER
I20260806 13:00:44.764403 13915 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb97270c4f16485bb748d17b51458ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34985 } }
I20260806 13:00:44.764503 13915 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:44.764547 13915 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:44.764591 13915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:44.765422 13915 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb97270c4f16485bb748d17b51458ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34985 } }
I20260806 13:00:44.765542 13915 leader_election.cc:302] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb97270c4f16485bb748d17b51458ece; no voters:
I20260806 13:00:44.765738 13915 leader_election.cc:288] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:44.765771 13920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:44.765875 13920 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [term 1 LEADER]: Becoming Leader. State: Replica: fb97270c4f16485bb748d17b51458ece, State: Running, Role: LEADER
I20260806 13:00:44.765995 13915 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:44.765987 13920 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb97270c4f16485bb748d17b51458ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34985 } }
I20260806 13:00:44.767160 13922 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb97270c4f16485bb748d17b51458ece. Latest consensus state: current_term: 1 leader_uuid: "fb97270c4f16485bb748d17b51458ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb97270c4f16485bb748d17b51458ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34985 } } }
I20260806 13:00:44.767284 13922 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:44.767402 13921 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb97270c4f16485bb748d17b51458ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb97270c4f16485bb748d17b51458ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34985 } } }
I20260806 13:00:44.767685 13921 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece [sys.catalog]: This master's current role is: LEADER
W20260806 13:00:44.767879 13934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:00:44.767964 13934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:00:44.768024 13935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:44.768306 13935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 13:00:44 dist-test-slave-zkpd krb5kdc[13831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 13:00:44.770828 13935 catalog_manager.cc:1347] Generated new cluster ID: fe28d39e80a54cb9ab3d88217d1db2c7
I20260806 13:00:44.770879 13935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:44.774914 13919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:44.764307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:55965 (local address 127.0.0.1:34985)
0806 13:00:44.764504 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:44.764515 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:44.764549 (+ 34us) server_negotiation.cc:407] Connection header received
0806 13:00:44.764688 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:44.764693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:44.764736 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:44.764869 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:44.765509 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:44.766660 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:44.767828 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:44.768456 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:44.768575 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:44.771187 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:44.771191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:44.771191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:44.771201 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:44.773189 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:44.773631 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:44.773634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:44.773634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:44.773760 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:44.773979 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:44.773980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:44.773980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:44.774248 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:44.774393 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:44.774625 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 13:00:44.774722 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260806 13:00:44.775882 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:44.784462 13919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:44.777460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:48683 (local address 127.0.0.1:34985)
0806 13:00:44.777490 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:44.777491 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:44.777499 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:00:44.777569 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:44.777569 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:44.777578 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:44.777626 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:44.778248 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:44.778954 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:44.779980 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:44.780293 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:44.780354 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:44.781705 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:44.781706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:44.781707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:44.781713 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:44.783238 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:44.783642 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:44.783643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:44.783643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:44.783694 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:44.783961 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:44.783962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:44.783962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:44.784084 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:44.784165 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:44.784319 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 13:00:44.784374 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 13:00:44.788185 13935 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:44.788928 13935 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:44.796316 13935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb97270c4f16485bb748d17b51458ece: Generated new TSK 0
I20260806 13:00:44.796476 13935 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:44.802870 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 13845
W20260806 13:00:44.808161 13829 connection.cc:535] client connection to 127.0.0.1:34985 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34985): Connection reset by peer (error 104)
2026-08-06T13:00:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:44 dist-test-slave-zkpd krb5kdc[13946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:44 dist-test-slave-zkpd krb5kdc[13946](info): set up 2 sockets
Aug 06 13:00:44 dist-test-slave-zkpd krb5kdc[13946](info): commencing operation
krb5kdc: starting...
W20260806 13:00:45.836520 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:45 dist-test-slave-zkpd krb5kdc[13946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:45.952252 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42031
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:46.032284 13959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:46.032481 13959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:46.032503 13959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:46.033573 13959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:46.033614 13959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:46.033628 13959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:46.033640 13959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:46.033653 13959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:46.035048 13959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:46.035238 13959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:46.035398 13959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:46.040146 13965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:46.040208 13966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:46.040175 13968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:46.040408 13959 server_base.cc:1034] running on GCE node
I20260806 13:00:46.040660 13959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:46.041002 13959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:46.042188 13959 hybrid_clock.cc:648] HybridClock initialized: now 1786021246042168 us; error 31 us; skew 500 ppm
Aug 06 13:00:46 dist-test-slave-zkpd krb5kdc[13946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021246, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:46.044700 13959 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:46.045141 13959 webserver.cc:460] Webserver started at http://127.0.0.1:38501/ using document root <none> and password file <none>
I20260806 13:00:46.045384 13959 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:46.045452 13959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:46.045598 13959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:46.046710 13959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "125746b3256548fa81c0906288b47e80"
format_stamp: "Formatted at 2026-08-06 13:00:46 on dist-test-slave-zkpd"
server_key: "f3a25540dfffc75e5a42a4e415c89d6f"
server_key_iv: "dab1aca5f4275fd0402652de2f66443c"
server_key_version: "encryptionkey@0"
I20260806 13:00:46.047106 13959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "125746b3256548fa81c0906288b47e80"
format_stamp: "Formatted at 2026-08-06 13:00:46 on dist-test-slave-zkpd"
server_key: "f3a25540dfffc75e5a42a4e415c89d6f"
server_key_iv: "dab1aca5f4275fd0402652de2f66443c"
server_key_version: "encryptionkey@0"
I20260806 13:00:46.048488 13959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 13:00:46.049252 13978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:46.049436 13959 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:46.049542 13959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "125746b3256548fa81c0906288b47e80"
format_stamp: "Formatted at 2026-08-06 13:00:46 on dist-test-slave-zkpd"
server_key: "f3a25540dfffc75e5a42a4e415c89d6f"
server_key_iv: "dab1aca5f4275fd0402652de2f66443c"
server_key_version: "encryptionkey@0"
I20260806 13:00:46.049615 13959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:46.062763 13959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:46.064651 13959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:46.064818 13959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:46.068799 13959 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42031
I20260806 13:00:46.068845 14034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42031 every 8 connection(s)
I20260806 13:00:46.069211 13959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:46.070081 14035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:46.072288 14035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80: Bootstrap starting.
I20260806 13:00:46.072803 14035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:46.073040 14035 log.cc:824] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:46.073695 14035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80: No bootstrap required, opened a new log
I20260806 13:00:46.075201 14035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125746b3256548fa81c0906288b47e80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42031 } }
I20260806 13:00:46.075332 14035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:46.075353 14035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125746b3256548fa81c0906288b47e80, State: Initialized, Role: FOLLOWER
I20260806 13:00:46.075484 14035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125746b3256548fa81c0906288b47e80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42031 } }
I20260806 13:00:46.075563 14035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:46.075587 14035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:46.075613 14035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:46.076144 14035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125746b3256548fa81c0906288b47e80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42031 } }
I20260806 13:00:46.076231 14035 leader_election.cc:302] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 125746b3256548fa81c0906288b47e80; no voters:
I20260806 13:00:46.076359 14035 leader_election.cc:288] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:46.076431 14038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:46.076618 14035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:46.076685 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 13959
I20260806 13:00:46.076803 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:46.076982 24908 external_mini_cluster.cc:1428] Setting key d9887f6af5d5ed7470688ece3fe2b745
I20260806 13:00:46.077147 14038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [term 1 LEADER]: Becoming Leader. State: Replica: 125746b3256548fa81c0906288b47e80, State: Running, Role: LEADER
I20260806 13:00:46.077309 14038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125746b3256548fa81c0906288b47e80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42031 } }
I20260806 13:00:46.077718 14041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 125746b3256548fa81c0906288b47e80. Latest consensus state: current_term: 1 leader_uuid: "125746b3256548fa81c0906288b47e80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125746b3256548fa81c0906288b47e80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42031 } } }
I20260806 13:00:46.077821 14041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:46.077921 14039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "125746b3256548fa81c0906288b47e80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125746b3256548fa81c0906288b47e80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42031 } } }
I20260806 13:00:46.078269 14039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:46.078497 14055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:46.078992 14055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:46.082685 14050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:46.077591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:49823 (local address 127.0.0.1:42031)
0806 13:00:46.077943 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:46.077952 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:46.077983 (+ 31us) server_negotiation.cc:407] Connection header received
0806 13:00:46.078150 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:46.078157 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:46.078218 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:46.078365 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:46.078633 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:46.079748 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:46.080839 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:46.081247 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:46.081323 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:46.082013 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:46.082017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:46.082017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:46.082186 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:46.082214 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:46.082224 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:46.082284 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:46.082438 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 13:00:46.082471 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":88,"threads_started":1}
I20260806 13:00:46.083828 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:46.088814 14050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:46.085373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:45257 (local address 127.0.0.1:42031)
0806 13:00:46.085414 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:46.085415 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:46.085487 (+ 72us) server_negotiation.cc:407] Connection header received
0806 13:00:46.085532 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:46.085532 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:46.085542 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:46.085613 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:46.085701 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:46.086752 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:46.087457 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:46.087796 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:46.087864 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:46.088571 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:46.088573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:46.088573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:46.088578 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:46.088591 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:46.088595 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:46.088668 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:46.088723 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 13:00:46.088736 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:00:46.089263 14055 catalog_manager.cc:1347] Generated new cluster ID: 8b15c2912cd54fc6989f97a1cd6f5c9e
I20260806 13:00:46.089318 14055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:46.110692 14055 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:46.111397 14055 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:46.116479 14055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 125746b3256548fa81c0906288b47e80: Generated new TSK 0
I20260806 13:00:46.116658 14055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:46.119825 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 13959
W20260806 13:00:46.125689 13944 connection.cc:535] client connection to 127.0.0.1:42031 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42031): Connection reset by peer (error 104)
2026-08-06T13:00:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:46 dist-test-slave-zkpd krb5kdc[14067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:46 dist-test-slave-zkpd krb5kdc[14067](info): set up 2 sockets
Aug 06 13:00:46 dist-test-slave-zkpd krb5kdc[14067](info): commencing operation
krb5kdc: starting...
W20260806 13:00:47.168481 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:47 dist-test-slave-zkpd krb5kdc[14067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:47.284972 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35573
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:47.366125 14082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:47.366307 14082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:47.366329 14082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:47.367422 14082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:47.367461 14082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:47.367475 14082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:47.367487 14082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:47.367542 14082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:47.369119 14082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:47.369356 14082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:47.369596 14082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:47.374253 14087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:47.374260 14088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:47.374254 14090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:47.374569 14082 server_base.cc:1034] running on GCE node
I20260806 13:00:47.374848 14082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:47.375164 14082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:47.376350 14082 hybrid_clock.cc:648] HybridClock initialized: now 1786021247376345 us; error 35 us; skew 500 ppm
Aug 06 13:00:47 dist-test-slave-zkpd krb5kdc[14067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021247, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:47.378728 14082 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:47.379212 14082 webserver.cc:460] Webserver started at http://127.0.0.1:36783/ using document root <none> and password file <none>
I20260806 13:00:47.379462 14082 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:47.379530 14082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:47.379683 14082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:47.380793 14082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "9e55c83674fc464d983e569cc238b0ae"
format_stamp: "Formatted at 2026-08-06 13:00:47 on dist-test-slave-zkpd"
server_key: "5a440913a1db7a7fd50adb6792c0c0c1"
server_key_iv: "486b2e7e8db1131d3daeda6b34b4062f"
server_key_version: "encryptionkey@0"
I20260806 13:00:47.381148 14082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "9e55c83674fc464d983e569cc238b0ae"
format_stamp: "Formatted at 2026-08-06 13:00:47 on dist-test-slave-zkpd"
server_key: "5a440913a1db7a7fd50adb6792c0c0c1"
server_key_iv: "486b2e7e8db1131d3daeda6b34b4062f"
server_key_version: "encryptionkey@0"
I20260806 13:00:47.382432 14082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:00:47.383150 14097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:47.383319 14082 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:00:47.383420 14082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "9e55c83674fc464d983e569cc238b0ae"
format_stamp: "Formatted at 2026-08-06 13:00:47 on dist-test-slave-zkpd"
server_key: "5a440913a1db7a7fd50adb6792c0c0c1"
server_key_iv: "486b2e7e8db1131d3daeda6b34b4062f"
server_key_version: "encryptionkey@0"
I20260806 13:00:47.383502 14082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:47.386747 14082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:47.388582 14082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:47.388729 14082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:47.392555 14082 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35573
I20260806 13:00:47.392583 14154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35573 every 8 connection(s)
I20260806 13:00:47.392884 14082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:47.393852 14155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:47.396057 14155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae: Bootstrap starting.
I20260806 13:00:47.396615 14155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:47.396898 14155 log.cc:824] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:47.397569 14155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae: No bootstrap required, opened a new log
I20260806 13:00:47.399137 14155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e55c83674fc464d983e569cc238b0ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35573 } }
I20260806 13:00:47.399288 14155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:47.399336 14155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e55c83674fc464d983e569cc238b0ae, State: Initialized, Role: FOLLOWER
I20260806 13:00:47.399439 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 14082
I20260806 13:00:47.399485 14155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e55c83674fc464d983e569cc238b0ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35573 } }
I20260806 13:00:47.399565 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:47.399585 14155 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:47.399624 14155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:47.399673 14155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:47.399750 24908 external_mini_cluster.cc:1428] Setting key 706e23398bf15055ff20f14db8eaeaeb
I20260806 13:00:47.400476 14155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e55c83674fc464d983e569cc238b0ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35573 } }
I20260806 13:00:47.400596 14155 leader_election.cc:302] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e55c83674fc464d983e569cc238b0ae; no voters:
I20260806 13:00:47.400786 14155 leader_election.cc:288] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:47.400852 14160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:47.400992 14160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [term 1 LEADER]: Becoming Leader. State: Replica: 9e55c83674fc464d983e569cc238b0ae, State: Running, Role: LEADER
I20260806 13:00:47.401054 14155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:47.401121 14160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e55c83674fc464d983e569cc238b0ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35573 } }
I20260806 13:00:47.401504 14161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e55c83674fc464d983e569cc238b0ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e55c83674fc464d983e569cc238b0ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35573 } } }
I20260806 13:00:47.401499 14160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e55c83674fc464d983e569cc238b0ae. Latest consensus state: current_term: 1 leader_uuid: "9e55c83674fc464d983e569cc238b0ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e55c83674fc464d983e569cc238b0ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35573 } } }
I20260806 13:00:47.401665 14161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:47.401674 14160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:47.401921 14164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:47.402513 14164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:47.402618 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:47.400617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:53561 (local address 127.0.0.1:35573)
0806 13:00:47.400791 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:47.400800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:47.400834 (+ 34us) server_negotiation.cc:407] Connection header received
0806 13:00:47.400950 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:47.400956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:47.400985 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:47.401443 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:47.401895 (+ 452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:47.401899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:47.401899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:47.401904 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:47.401950 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:47.401967 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:47.402303 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:47.402335 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 13:00:47.402380 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260806 13:00:47.403606 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:47.406085 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:47.404987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:57693 (local address 127.0.0.1:35573)
0806 13:00:47.405034 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:47.405036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:47.405166 (+ 130us) server_negotiation.cc:407] Connection header received
0806 13:00:47.405238 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:47.405239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:47.405243 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:47.405306 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:47.405879 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:47.405880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:47.405881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:47.405885 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:47.405899 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:47.405903 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:47.405954 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:47.406007 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 13:00:47.406021 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 13:00:47.408242 14164 catalog_manager.cc:1347] Generated new cluster ID: c0b8b6cbdff547abbded93356750b80b
I20260806 13:00:47.408308 14164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:47.435238 14164 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:47.435433 14164 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:47.442006 14164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e55c83674fc464d983e569cc238b0ae: Generated new TSK 0
I20260806 13:00:47.442186 14164 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:47.451463 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 14082
2026-08-06T13:00:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:47 dist-test-slave-zkpd krb5kdc[14187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:47 dist-test-slave-zkpd krb5kdc[14187](info): set up 2 sockets
Aug 06 13:00:47 dist-test-slave-zkpd krb5kdc[14187](info): commencing operation
krb5kdc: starting...
W20260806 13:00:48.488843 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:48 dist-test-slave-zkpd krb5kdc[14187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:48.602180 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45085
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:48.681226 14200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:48.681433 14200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:48.681468 14200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:48.682608 14200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:48.682677 14200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:48.682701 14200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:48.682729 14200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:48.682770 14200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:48.684247 14200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45085
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:48.684525 14200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:48.684741 14200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:48.689396 14206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:48.689385 14208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:48.689484 14200 server_base.cc:1034] running on GCE node
W20260806 13:00:48.689385 14205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:48.689848 14200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:48.690135 14200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:48.691299 14200 hybrid_clock.cc:648] HybridClock initialized: now 1786021248691294 us; error 36 us; skew 500 ppm
Aug 06 13:00:48 dist-test-slave-zkpd krb5kdc[14187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:48.698909 14200 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:48.699317 14200 webserver.cc:460] Webserver started at http://127.0.0.1:33815/ using document root <none> and password file <none>
I20260806 13:00:48.699543 14200 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:48.699611 14200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:48.699759 14200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:48.700851 14200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "f4a0443a19594df580a3ec65321811ff"
format_stamp: "Formatted at 2026-08-06 13:00:48 on dist-test-slave-zkpd"
server_key: "a945476006d1e38f6cfdb03112b59469"
server_key_iv: "eb65aae4f55d35f7d56b45beb02edafb"
server_key_version: "encryptionkey@0"
I20260806 13:00:48.701213 14200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "f4a0443a19594df580a3ec65321811ff"
format_stamp: "Formatted at 2026-08-06 13:00:48 on dist-test-slave-zkpd"
server_key: "a945476006d1e38f6cfdb03112b59469"
server_key_iv: "eb65aae4f55d35f7d56b45beb02edafb"
server_key_version: "encryptionkey@0"
I20260806 13:00:48.702508 14200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:00:48.703222 14215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:48.703390 14200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:48.703487 14200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "f4a0443a19594df580a3ec65321811ff"
format_stamp: "Formatted at 2026-08-06 13:00:48 on dist-test-slave-zkpd"
server_key: "a945476006d1e38f6cfdb03112b59469"
server_key_iv: "eb65aae4f55d35f7d56b45beb02edafb"
server_key_version: "encryptionkey@0"
I20260806 13:00:48.703572 14200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:48.737401 14200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:48.739769 14200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:48.739962 14200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:48.743955 14269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45085 every 8 connection(s)
I20260806 13:00:48.743959 14200 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45085
I20260806 13:00:48.744371 14200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:48.745213 14270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:48.746764 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 14200
I20260806 13:00:48.746883 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:48.747072 24908 external_mini_cluster.cc:1428] Setting key 836f6d4a2cfbc9a546d79a1b389fbe43
I20260806 13:00:48.747658 14270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff: Bootstrap starting.
I20260806 13:00:48.748353 14270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:48.748661 14270 log.cc:824] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:48.749351 14270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff: No bootstrap required, opened a new log
I20260806 13:00:48.751081 14270 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a0443a19594df580a3ec65321811ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45085 } }
I20260806 13:00:48.751235 14270 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:48.751289 14270 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4a0443a19594df580a3ec65321811ff, State: Initialized, Role: FOLLOWER
I20260806 13:00:48.751475 14270 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a0443a19594df580a3ec65321811ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45085 } }
I20260806 13:00:48.751595 14270 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:48.751636 14270 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:48.751684 14270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:48.752429 14270 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a0443a19594df580a3ec65321811ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45085 } }
I20260806 13:00:48.752565 14270 leader_election.cc:302] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4a0443a19594df580a3ec65321811ff; no voters:
I20260806 13:00:48.752745 14270 leader_election.cc:288] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:48.752807 14276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:48.752976 14276 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [term 1 LEADER]: Becoming Leader. State: Replica: f4a0443a19594df580a3ec65321811ff, State: Running, Role: LEADER
I20260806 13:00:48.752988 14270 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:48.753134 14276 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a0443a19594df580a3ec65321811ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45085 } }
Aug 06 13:00:48 dist-test-slave-zkpd krb5kdc[14187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 13:00:48.754082 14275 raft_consensus.cc:465] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [term 1 LEADER]: Not starting pre-election -- already a leader
I20260806 13:00:48.754444 14277 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4a0443a19594df580a3ec65321811ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a0443a19594df580a3ec65321811ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45085 } } }
I20260806 13:00:48.754557 14278 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4a0443a19594df580a3ec65321811ff. Latest consensus state: current_term: 1 leader_uuid: "f4a0443a19594df580a3ec65321811ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a0443a19594df580a3ec65321811ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45085 } } }
I20260806 13:00:48.754913 14277 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:48.754914 14278 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff [sys.catalog]: This master's current role is: LEADER
W20260806 13:00:48.755092 14294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:00:48.755134 14294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:00:48.755188 14296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:48.755367 14296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:48.756826 14296 catalog_manager.cc:1347] Generated new cluster ID: ca7d3097289a45909d8b5b13b5918d70
I20260806 13:00:48.756918 14296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:48.757967 14273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:48.747597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:59561 (local address 127.0.0.1:45085)
0806 13:00:48.747782 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:48.747790 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:48.747815 (+ 25us) server_negotiation.cc:407] Connection header received
0806 13:00:48.748282 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:48.748288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:48.748337 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:48.748487 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:48.749173 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:48.750094 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:48.750871 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:48.751296 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:48.751409 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:48.754330 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:48.754334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:48.754335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:48.754346 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:48.756266 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:48.756680 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:48.756683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:48.756684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:48.756799 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:48.757062 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:48.757063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:48.757063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:48.757362 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:48.757524 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:48.757664 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 13:00:48.757775 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260806 13:00:48.758859 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:48.762424 14296 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:48.763082 14296 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:48.774032 14273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:48.764168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:54147 (local address 127.0.0.1:45085)
0806 13:00:48.764207 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:48.764209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:48.768066 (+ 3857us) server_negotiation.cc:407] Connection header received
0806 13:00:48.768140 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:48.768141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:48.768153 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:48.768221 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:48.768731 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:48.769220 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:48.769978 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:48.770203 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:48.770279 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:48.771512 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:48.771514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:48.771514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:48.771521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:48.772892 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:48.773247 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:48.773248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:48.773248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:48.773305 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:48.773527 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:48.773528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:48.773528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:48.773630 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:48.773757 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:48.773863 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 13:00:48.773927 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:00:48.778154 14296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4a0443a19594df580a3ec65321811ff: Generated new TSK 0
I20260806 13:00:48.778323 14296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:48.781412 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 14200
W20260806 13:00:48.787201 14185 connection.cc:535] client connection to 127.0.0.1:45085 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45085): Connection reset by peer (error 104)
2026-08-06T13:00:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1330 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:48 dist-test-slave-zkpd krb5kdc[14305](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:48 dist-test-slave-zkpd krb5kdc[14305](info): set up 2 sockets
Aug 06 13:00:48 dist-test-slave-zkpd krb5kdc[14305](info): commencing operation
krb5kdc: starting...
W20260806 13:00:49.834580 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:49 dist-test-slave-zkpd krb5kdc[14305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:49.947120 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:50.027210 14319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:50.027467 14319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:50.027506 14319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:50.028657 14319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:50.028730 14319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:50.028755 14319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:50.028777 14319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:50.028807 14319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:50.030237 14319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46055
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:50.030516 14319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:50.030756 14319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:50.035498 14325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:50.035567 14319 server_base.cc:1034] running on GCE node
W20260806 13:00:50.035482 14324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:50.035461 14327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:50.035863 14319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:50.036203 14319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:50.037375 14319 hybrid_clock.cc:648] HybridClock initialized: now 1786021250037360 us; error 26 us; skew 500 ppm
Aug 06 13:00:50 dist-test-slave-zkpd krb5kdc[14305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021250, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:50.039849 14319 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:50.040326 14319 webserver.cc:460] Webserver started at http://127.0.0.1:43999/ using document root <none> and password file <none>
I20260806 13:00:50.040548 14319 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:50.040613 14319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:50.040776 14319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:50.041812 14319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "cc25c701889642e7ab4ee8be533025a9"
format_stamp: "Formatted at 2026-08-06 13:00:50 on dist-test-slave-zkpd"
server_key: "5a72fd60abf228e89336db8f6692d445"
server_key_iv: "c53905f10c76d21495873bea189732c7"
server_key_version: "encryptionkey@0"
I20260806 13:00:50.042172 14319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "cc25c701889642e7ab4ee8be533025a9"
format_stamp: "Formatted at 2026-08-06 13:00:50 on dist-test-slave-zkpd"
server_key: "5a72fd60abf228e89336db8f6692d445"
server_key_iv: "c53905f10c76d21495873bea189732c7"
server_key_version: "encryptionkey@0"
I20260806 13:00:50.043531 14319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:00:50.044317 14335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:50.044483 14319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:50.044582 14319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "cc25c701889642e7ab4ee8be533025a9"
format_stamp: "Formatted at 2026-08-06 13:00:50 on dist-test-slave-zkpd"
server_key: "5a72fd60abf228e89336db8f6692d445"
server_key_iv: "c53905f10c76d21495873bea189732c7"
server_key_version: "encryptionkey@0"
I20260806 13:00:50.044663 14319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:50.056959 14319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:50.059484 14319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:50.059648 14319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:50.063733 14319 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46055
I20260806 13:00:50.063764 14389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46055 every 8 connection(s)
I20260806 13:00:50.064185 14319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:50.065006 14390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:50.067209 14390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9: Bootstrap starting.
I20260806 13:00:50.067776 14390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:50.068095 14390 log.cc:824] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:50.068758 14390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9: No bootstrap required, opened a new log
I20260806 13:00:50.070290 14390 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc25c701889642e7ab4ee8be533025a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46055 } }
I20260806 13:00:50.070425 14390 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:50.070446 14390 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc25c701889642e7ab4ee8be533025a9, State: Initialized, Role: FOLLOWER
I20260806 13:00:50.070655 14390 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc25c701889642e7ab4ee8be533025a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46055 } }
I20260806 13:00:50.070752 14390 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:50.070786 14390 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:50.070844 14390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:50.071383 14390 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc25c701889642e7ab4ee8be533025a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46055 } }
I20260806 13:00:50.071416 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 14319
I20260806 13:00:50.071524 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:50.071503 14390 leader_election.cc:302] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc25c701889642e7ab4ee8be533025a9; no voters:
I20260806 13:00:50.071669 14390 leader_election.cc:288] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:50.071719 14393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:50.071750 24908 external_mini_cluster.cc:1428] Setting key 7058d74a81d802c2b91cf1a54cb8fe6f
I20260806 13:00:50.071869 14393 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [term 1 LEADER]: Becoming Leader. State: Replica: cc25c701889642e7ab4ee8be533025a9, State: Running, Role: LEADER
I20260806 13:00:50.071949 14390 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:50.072000 14393 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc25c701889642e7ab4ee8be533025a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46055 } }
I20260806 13:00:50.072693 14394 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc25c701889642e7ab4ee8be533025a9. Latest consensus state: current_term: 1 leader_uuid: "cc25c701889642e7ab4ee8be533025a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc25c701889642e7ab4ee8be533025a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46055 } } }
I20260806 13:00:50.072834 14394 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:50.073028 14395 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc25c701889642e7ab4ee8be533025a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc25c701889642e7ab4ee8be533025a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46055 } } }
I20260806 13:00:50.073153 14395 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:50.073230 14404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:50.073835 14404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:50.075392 14404 catalog_manager.cc:1347] Generated new cluster ID: 65ba1861349c4987adec4c55d7dbc976
I20260806 13:00:50.075446 14404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:50.077701 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:50.072214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:40875 (local address 127.0.0.1:46055)
0806 13:00:50.072438 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:50.072446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:50.072479 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:00:50.072592 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:50.072599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:50.072663 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:50.073100 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:50.073217 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:50.074744 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:50.075738 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:50.076204 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:50.076330 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:50.077030 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:50.077034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:50.077035 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:50.077208 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:50.077237 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:50.077248 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:50.077339 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:50.077423 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 13:00:50.077541 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":68,"threads_started":1}
I20260806 13:00:50.078569 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:50.086928 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:50.079886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:38929 (local address 127.0.0.1:46055)
0806 13:00:50.079939 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:50.079940 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:50.084136 (+ 4196us) server_negotiation.cc:407] Connection header received
0806 13:00:50.084211 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:50.084212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:50.084223 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:50.084278 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:50.084440 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:50.084925 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:50.085644 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:50.085872 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:50.085954 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:50.086676 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:50.086678 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:50.086678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:50.086682 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:50.086696 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:50.086700 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:50.086752 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:50.086814 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 13:00:50.086825 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 13:00:50.111714 14404 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:50.112317 14404 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:50.119881 14404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc25c701889642e7ab4ee8be533025a9: Generated new TSK 0
I20260806 13:00:50.120113 14404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:50.124963 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 14319
W20260806 13:00:50.130565 14303 connection.cc:535] client connection to 127.0.0.1:46055 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46055): Connection reset by peer (error 104)
2026-08-06T13:00:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:50 dist-test-slave-zkpd krb5kdc[14422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:50 dist-test-slave-zkpd krb5kdc[14422](info): set up 2 sockets
Aug 06 13:00:50 dist-test-slave-zkpd krb5kdc[14422](info): commencing operation
krb5kdc: starting...
W20260806 13:00:51.164376 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:51 dist-test-slave-zkpd krb5kdc[14422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:00:51.276701 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42549
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:51.357016 14440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:51.357210 14440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:51.357232 14440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:51.358289 14440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:51.358323 14440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:51.358337 14440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:51.358350 14440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:51.358364 14440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:51.359711 14440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42549
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:51.359920 14440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:51.360127 14440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:51.364737 14445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:51.364748 14448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:51.364965 14440 server_base.cc:1034] running on GCE node
W20260806 13:00:51.364748 14446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:51.365229 14440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:51.365521 14440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:51.366688 14440 hybrid_clock.cc:648] HybridClock initialized: now 1786021251366679 us; error 48 us; skew 500 ppm
Aug 06 13:00:51 dist-test-slave-zkpd krb5kdc[14422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021251, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:51.369108 14440 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:00:51.369578 14440 webserver.cc:460] Webserver started at http://127.0.0.1:38781/ using document root <none> and password file <none>
I20260806 13:00:51.369793 14440 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:51.369872 14440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:51.370019 14440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:51.371064 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "2f5ba004b78947308535d52708008189"
format_stamp: "Formatted at 2026-08-06 13:00:51 on dist-test-slave-zkpd"
server_key: "105f2c1be8539e46f02db4aab408419d"
server_key_iv: "3da84b9859cd67a6014d9ac083b63ab3"
server_key_version: "encryptionkey@0"
I20260806 13:00:51.371423 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "2f5ba004b78947308535d52708008189"
format_stamp: "Formatted at 2026-08-06 13:00:51 on dist-test-slave-zkpd"
server_key: "105f2c1be8539e46f02db4aab408419d"
server_key_iv: "3da84b9859cd67a6014d9ac083b63ab3"
server_key_version: "encryptionkey@0"
I20260806 13:00:51.372802 14440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:00:51.373482 14455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:51.373689 14440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:51.373798 14440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "2f5ba004b78947308535d52708008189"
format_stamp: "Formatted at 2026-08-06 13:00:51 on dist-test-slave-zkpd"
server_key: "105f2c1be8539e46f02db4aab408419d"
server_key_iv: "3da84b9859cd67a6014d9ac083b63ab3"
server_key_version: "encryptionkey@0"
I20260806 13:00:51.373880 14440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:51.377163 14440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:51.378913 14440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:51.379072 14440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:51.383189 14440 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42549
I20260806 13:00:51.383236 14507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42549 every 8 connection(s)
I20260806 13:00:51.383558 14440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:51.384563 14508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:51.386673 14508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189: Bootstrap starting.
I20260806 13:00:51.387239 14508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:51.387530 14508 log.cc:824] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:51.388262 14508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189: No bootstrap required, opened a new log
I20260806 13:00:51.389803 14508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5ba004b78947308535d52708008189" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42549 } }
I20260806 13:00:51.389988 14508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:51.390033 14508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f5ba004b78947308535d52708008189, State: Initialized, Role: FOLLOWER
I20260806 13:00:51.390192 14508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5ba004b78947308535d52708008189" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42549 } }
I20260806 13:00:51.390254 14508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:51.390276 14508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:51.390341 14508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:51.390870 14508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5ba004b78947308535d52708008189" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42549 } }
I20260806 13:00:51.390905 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 14440
I20260806 13:00:51.390988 14508 leader_election.cc:302] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f5ba004b78947308535d52708008189; no voters:
I20260806 13:00:51.391022 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:51.391182 14508 leader_election.cc:288] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:51.391215 24908 external_mini_cluster.cc:1428] Setting key 3a750631c279b46cda079e809e226bb7
I20260806 13:00:51.391208 14511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:51.391388 14511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [term 1 LEADER]: Becoming Leader. State: Replica: 2f5ba004b78947308535d52708008189, State: Running, Role: LEADER
I20260806 13:00:51.391494 14508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:51.391528 14511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5ba004b78947308535d52708008189" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42549 } }
I20260806 13:00:51.391830 14511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f5ba004b78947308535d52708008189. Latest consensus state: current_term: 1 leader_uuid: "2f5ba004b78947308535d52708008189" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5ba004b78947308535d52708008189" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42549 } } }
I20260806 13:00:51.391929 14511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:51.392067 14512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f5ba004b78947308535d52708008189" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5ba004b78947308535d52708008189" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42549 } } }
I20260806 13:00:51.392198 14512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:51.392854 14517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:51.393404 14517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:51.393666 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:51.391836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:58845 (local address 127.0.0.1:42549)
0806 13:00:51.392131 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:51.392139 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:51.392171 (+ 32us) server_negotiation.cc:407] Connection header received
0806 13:00:51.392303 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:51.392311 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:51.392344 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:51.392505 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:51.393138 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:51.393142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:51.393142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:51.393146 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:51.393188 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:51.393205 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:51.393421 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:51.393451 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 13:00:51.393492 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":93,"threads_started":1}
I20260806 13:00:51.394827 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:51.396564 14517 catalog_manager.cc:1347] Generated new cluster ID: 89c9afd5dabb4ed18db049cc44b14b73
I20260806 13:00:51.396617 14517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:51.397598 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:51.396407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:41435 (local address 127.0.0.1:42549)
0806 13:00:51.396440 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:51.396442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:51.396603 (+ 161us) server_negotiation.cc:407] Connection header received
0806 13:00:51.396662 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:51.396663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:51.396668 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:51.396735 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:51.397364 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:51.397365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:51.397366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:51.397370 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:51.397394 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:51.397399 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:51.397460 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:51.397511 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 13:00:51.397524 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:00:51.415201 14517 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:51.415380 14517 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:51.418642 14517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f5ba004b78947308535d52708008189: Generated new TSK 0
I20260806 13:00:51.418820 14517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:51.422477 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 14440
2026-08-06T13:00:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:51 dist-test-slave-zkpd krb5kdc[14539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:51 dist-test-slave-zkpd krb5kdc[14539](info): set up 2 sockets
Aug 06 13:00:51 dist-test-slave-zkpd krb5kdc[14539](info): commencing operation
krb5kdc: starting...
W20260806 13:00:52.464406 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:52 dist-test-slave-zkpd krb5kdc[14539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 13:00:52.578343 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:35357
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:46641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:35357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:52.658074 14552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:52.658308 14552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:52.658345 14552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:52.659634 14552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:52.659698 14552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:52.659723 14552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:52.659756 14552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:52.659785 14552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:52.661242 14552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:46641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:35357
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:35357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:52.661526 14552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:52.661746 14552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 13:00:52.666457 14552 server_base.cc:1034] running on GCE node
W20260806 13:00:52.666430 14559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:52.666430 14558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:52.666532 14561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:52.666844 14552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:52.667161 14552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:52.668310 14552 hybrid_clock.cc:648] HybridClock initialized: now 1786021252668289 us; error 36 us; skew 500 ppm
Aug 06 13:00:52 dist-test-slave-zkpd krb5kdc[14539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021252, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:52.670636 14552 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:00:52.671034 14552 webserver.cc:460] Webserver started at http://127.24.83.62:39137/ using document root <none> and password file <none>
I20260806 13:00:52.671243 14552 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:52.671309 14552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:52.671454 14552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:52.672490 14552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "b792761285064b99857129eb3a9a469e"
format_stamp: "Formatted at 2026-08-06 13:00:52 on dist-test-slave-zkpd"
server_key: "7da4a8ec05f245d7e58c7fcae350220f"
server_key_iv: "5f256dc10e27aea6dbe68ee2951f851b"
server_key_version: "encryptionkey@0"
I20260806 13:00:52.672847 14552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "b792761285064b99857129eb3a9a469e"
format_stamp: "Formatted at 2026-08-06 13:00:52 on dist-test-slave-zkpd"
server_key: "7da4a8ec05f245d7e58c7fcae350220f"
server_key_iv: "5f256dc10e27aea6dbe68ee2951f851b"
server_key_version: "encryptionkey@0"
I20260806 13:00:52.674125 14552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:00:52.674831 14568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:52.675021 14552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:52.675124 14552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "b792761285064b99857129eb3a9a469e"
format_stamp: "Formatted at 2026-08-06 13:00:52 on dist-test-slave-zkpd"
server_key: "7da4a8ec05f245d7e58c7fcae350220f"
server_key_iv: "5f256dc10e27aea6dbe68ee2951f851b"
server_key_version: "encryptionkey@0"
I20260806 13:00:52.675194 14552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:52.684213 14552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:52.685861 14552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:52.686015 14552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:52.690042 14552 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:35357
I20260806 13:00:52.690073 14625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:35357 every 8 connection(s)
I20260806 13:00:52.690397 14552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:52.691164 14626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:52.692955 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 14552
I20260806 13:00:52.693075 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:52.693246 24908 external_mini_cluster.cc:1428] Setting key 578e82c62fd86ffdcfa655e0c97a0825
I20260806 13:00:52.693692 14626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e: Bootstrap starting.
I20260806 13:00:52.694332 14626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:52.694665 14626 log.cc:824] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:52.695400 14626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e: No bootstrap required, opened a new log
I20260806 13:00:52.697576 14626 raft_consensus.cc:348] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792761285064b99857129eb3a9a469e" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35357 } }
I20260806 13:00:52.697702 14626 raft_consensus.cc:374] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:52.697726 14626 raft_consensus.cc:729] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b792761285064b99857129eb3a9a469e, State: Initialized, Role: FOLLOWER
I20260806 13:00:52.697860 14626 consensus_queue.cc:260] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792761285064b99857129eb3a9a469e" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35357 } }
I20260806 13:00:52.697921 14626 raft_consensus.cc:388] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:52.697942 14626 raft_consensus.cc:482] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:52.698007 14626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:52.698751 14626 raft_consensus.cc:504] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792761285064b99857129eb3a9a469e" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35357 } }
I20260806 13:00:52.698871 14626 leader_election.cc:302] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b792761285064b99857129eb3a9a469e; no voters:
I20260806 13:00:52.699035 14626 leader_election.cc:288] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:52.699092 14631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:52.699249 14631 raft_consensus.cc:686] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [term 1 LEADER]: Becoming Leader. State: Replica: b792761285064b99857129eb3a9a469e, State: Running, Role: LEADER
I20260806 13:00:52.699301 14626 sys_catalog.cc:564] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:52.699378 14631 consensus_queue.cc:237] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792761285064b99857129eb3a9a469e" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35357 } }
I20260806 13:00:52.699673 14631 sys_catalog.cc:455] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b792761285064b99857129eb3a9a469e. Latest consensus state: current_term: 1 leader_uuid: "b792761285064b99857129eb3a9a469e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792761285064b99857129eb3a9a469e" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35357 } } }
I20260806 13:00:52.699692 14633 sys_catalog.cc:455] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b792761285064b99857129eb3a9a469e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792761285064b99857129eb3a9a469e" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 35357 } } }
I20260806 13:00:52.699827 14631 sys_catalog.cc:458] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:52.699836 14633 sys_catalog.cc:458] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:52.700119 14637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 13:00:52 dist-test-slave-zkpd krb5kdc[14539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:52.701155 14637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:52.703644 14637 catalog_manager.cc:1347] Generated new cluster ID: 4c81a265e3fb4e779c66210caffbc200
I20260806 13:00:52.703699 14637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:52.704576 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:52.693759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51943 (local address 127.24.83.62:35357)
0806 13:00:52.693922 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:52.693930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:52.693953 (+ 23us) server_negotiation.cc:407] Connection header received
0806 13:00:52.694347 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:52.694353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:52.694429 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:52.694543 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:52.695325 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:52.696381 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:52.697535 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:52.698089 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:52.698176 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:52.701043 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:52.701047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:52.701048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:52.701059 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:52.702771 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:52.703216 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:52.703219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:52.703219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:52.703335 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:52.703597 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:52.703598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:52.703598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:52.703959 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:52.704128 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:52.704262 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 13:00:52.704364 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260806 13:00:52.705492 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:52.717970 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:52.708247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34363 (local address 127.24.83.62:35357)
0806 13:00:52.708287 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:52.708289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:52.712047 (+ 3758us) server_negotiation.cc:407] Connection header received
0806 13:00:52.712117 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:52.712118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:52.712127 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:52.712178 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:52.712665 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:52.713134 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:52.713831 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:52.714069 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:52.714155 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:52.715320 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:52.715321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:52.715322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:52.715329 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:52.716792 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:52.717167 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:52.717168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:52.717169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:52.717222 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:52.717450 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:52.717451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:52.717451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:52.717558 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:52.717663 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:52.717793 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 13:00:52.717856 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:00:52.736279 14637 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:52.736984 14637 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:52.739780 14637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b792761285064b99857129eb3a9a469e: Generated new TSK 0
I20260806 13:00:52.739975 14637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:52.742872 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 14552
W20260806 13:00:52.748026 14537 connection.cc:535] client connection to 127.24.83.62:35357 recv error: Network error: failed to read from TLS socket (remote: 127.24.83.62:35357): Connection reset by peer (error 104)
2026-08-06T13:00:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:52 dist-test-slave-zkpd krb5kdc[14657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:52 dist-test-slave-zkpd krb5kdc[14657](info): set up 2 sockets
Aug 06 13:00:52 dist-test-slave-zkpd krb5kdc[14657](info): commencing operation
krb5kdc: starting...
W20260806 13:00:53.776623 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:53 dist-test-slave-zkpd krb5kdc[14657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 13:00:53.888868 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:44903
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:44903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:53.968349 14670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:53.968576 14670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:53.968616 14670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:53.969734 14670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:53.969803 14670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:53.969827 14670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:53.969857 14670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:53.969898 14670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:53.971396 14670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:44903
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:44903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:53.971662 14670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:53.971909 14670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:53.976689 14676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:53.976696 14678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:53.976697 14675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:53.977020 14670 server_base.cc:1034] running on GCE node
I20260806 13:00:53.977279 14670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:53.977550 14670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:53.978708 14670 hybrid_clock.cc:648] HybridClock initialized: now 1786021253978698 us; error 39 us; skew 500 ppm
Aug 06 13:00:53 dist-test-slave-zkpd krb5kdc[14657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021253, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:53.986321 14670 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:00:53.986685 14670 webserver.cc:460] Webserver started at http://127.24.83.62:42169/ using document root <none> and password file <none>
I20260806 13:00:53.986856 14670 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:53.986894 14670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:53.986989 14670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:53.988042 14670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "617f0209352749bc82797dd49b2ec965"
format_stamp: "Formatted at 2026-08-06 13:00:53 on dist-test-slave-zkpd"
server_key: "3b06c4fbd3ccb1bc4dd6bc3204eb36fc"
server_key_iv: "81c3324eb60f75740a28d740596ae53d"
server_key_version: "encryptionkey@0"
I20260806 13:00:53.988348 14670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "617f0209352749bc82797dd49b2ec965"
format_stamp: "Formatted at 2026-08-06 13:00:53 on dist-test-slave-zkpd"
server_key: "3b06c4fbd3ccb1bc4dd6bc3204eb36fc"
server_key_iv: "81c3324eb60f75740a28d740596ae53d"
server_key_version: "encryptionkey@0"
I20260806 13:00:53.989598 14670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:53.990329 14685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:53.990531 14670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:53.990605 14670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "617f0209352749bc82797dd49b2ec965"
format_stamp: "Formatted at 2026-08-06 13:00:53 on dist-test-slave-zkpd"
server_key: "3b06c4fbd3ccb1bc4dd6bc3204eb36fc"
server_key_iv: "81c3324eb60f75740a28d740596ae53d"
server_key_version: "encryptionkey@0"
I20260806 13:00:53.990689 14670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:54.014294 14670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:54.016635 14670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:54.016794 14670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:54.020740 14670 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:44903
I20260806 13:00:54.020921 14741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:44903 every 8 connection(s)
I20260806 13:00:54.021131 14670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:54.021957 14742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:54.023159 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 14670
I20260806 13:00:54.023293 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:54.023516 24908 external_mini_cluster.cc:1428] Setting key 112ceed1f9e69b9667fc96182ec11cd6
I20260806 13:00:54.024562 14742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965: Bootstrap starting.
I20260806 13:00:54.025300 14742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:54.025599 14742 log.cc:824] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:54.026373 14742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965: No bootstrap required, opened a new log
I20260806 13:00:54.028723 14742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617f0209352749bc82797dd49b2ec965" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44903 } }
I20260806 13:00:54.028909 14742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:54.028959 14742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 617f0209352749bc82797dd49b2ec965, State: Initialized, Role: FOLLOWER
I20260806 13:00:54.029004 14746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:54.024046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36529 (local address 127.24.83.62:44903)
0806 13:00:54.024233 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:54.024241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:54.024277 (+ 36us) server_negotiation.cc:407] Connection header received
0806 13:00:54.024690 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:54.024700 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:54.024757 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:54.024875 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:54.025152 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:54.026111 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:54.027164 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:54.027572 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:54.027650 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:54.028353 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:54.028356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:54.028357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:54.028508 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:54.028537 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:54.028547 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:54.028610 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:54.028724 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 13:00:54.028766 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260806 13:00:54.029084 14742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617f0209352749bc82797dd49b2ec965" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44903 } }
I20260806 13:00:54.029168 14742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:54.029207 14742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:54.029254 14742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:54.029968 14742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617f0209352749bc82797dd49b2ec965" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44903 } }
I20260806 13:00:54.030030 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:54.030143 14742 leader_election.cc:302] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 617f0209352749bc82797dd49b2ec965; no voters:
I20260806 13:00:54.030344 14742 leader_election.cc:288] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:54.030380 14749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:54.030571 14749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [term 1 LEADER]: Becoming Leader. State: Replica: 617f0209352749bc82797dd49b2ec965, State: Running, Role: LEADER
I20260806 13:00:54.030615 14742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:54.030735 14749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617f0209352749bc82797dd49b2ec965" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44903 } }
I20260806 13:00:54.031078 14749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 617f0209352749bc82797dd49b2ec965. Latest consensus state: current_term: 1 leader_uuid: "617f0209352749bc82797dd49b2ec965" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617f0209352749bc82797dd49b2ec965" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44903 } } }
I20260806 13:00:54.031319 14749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:54.031250 14752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "617f0209352749bc82797dd49b2ec965" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617f0209352749bc82797dd49b2ec965" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 44903 } } }
I20260806 13:00:54.031710 14752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:54.031839 14760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:54.032433 14760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:54.034166 14760 catalog_manager.cc:1347] Generated new cluster ID: 8a4ebdd7349e4a62b0c25cd144553335
I20260806 13:00:54.034225 14760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:54.035523 14746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:54.032079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58833 (local address 127.24.83.62:44903)
0806 13:00:54.032116 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:54.032118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:54.032222 (+ 104us) server_negotiation.cc:407] Connection header received
0806 13:00:54.032317 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:54.032318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:54.032328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:54.032357 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:54.032564 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:54.033275 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:54.034225 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:54.034507 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:54.034546 (+ 39us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:54.035290 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:54.035291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:54.035291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:54.035295 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:54.035307 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:54.035311 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:54.035340 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:54.035448 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 13:00:54.035457 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:00:54.065400 14760 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:54.066104 14760 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:54.070704 14760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 617f0209352749bc82797dd49b2ec965: Generated new TSK 0
I20260806 13:00:54.070886 14760 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:54.073817 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 14670
W20260806 13:00:54.079638 14655 connection.cc:535] client connection to 127.24.83.62:44903 recv error: Network error: failed to read from TLS socket (remote: 127.24.83.62:44903): Connection reset by peer (error 104)
2026-08-06T13:00:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:54 dist-test-slave-zkpd krb5kdc[14776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:54 dist-test-slave-zkpd krb5kdc[14776](info): set up 2 sockets
Aug 06 13:00:54 dist-test-slave-zkpd krb5kdc[14776](info): commencing operation
krb5kdc: starting...
W20260806 13:00:55.131927 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:55 dist-test-slave-zkpd krb5kdc[14776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 13:00:55.250083 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:46677
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:46677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:55.331416 14789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:55.331606 14789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:55.331629 14789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:55.332799 14789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:55.332842 14789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:55.332856 14789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:55.332870 14789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:55.332881 14789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:55.334383 14789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:46677
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:46677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:55.334643 14789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:55.334870 14789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:55.339624 14795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:55.339636 14797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:55.339632 14794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:55.339766 14789 server_base.cc:1034] running on GCE node
I20260806 13:00:55.340077 14789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:55.340410 14789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:55.341557 14789 hybrid_clock.cc:648] HybridClock initialized: now 1786021255341546 us; error 26 us; skew 500 ppm
Aug 06 13:00:55 dist-test-slave-zkpd krb5kdc[14776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:55.344022 14789 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:00:55.344501 14789 webserver.cc:460] Webserver started at http://127.24.83.62:37169/ using document root <none> and password file <none>
I20260806 13:00:55.344725 14789 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:55.344795 14789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:55.344930 14789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:55.345964 14789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "c34ffbe9a9634632bdf6fe9a98f08393"
format_stamp: "Formatted at 2026-08-06 13:00:55 on dist-test-slave-zkpd"
server_key: "8ca904da9e590f2a2b463963aa009f8b"
server_key_iv: "feaf04775fdfe5ba6dbabde3f86514d0"
server_key_version: "encryptionkey@0"
I20260806 13:00:55.346326 14789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "c34ffbe9a9634632bdf6fe9a98f08393"
format_stamp: "Formatted at 2026-08-06 13:00:55 on dist-test-slave-zkpd"
server_key: "8ca904da9e590f2a2b463963aa009f8b"
server_key_iv: "feaf04775fdfe5ba6dbabde3f86514d0"
server_key_version: "encryptionkey@0"
I20260806 13:00:55.347641 14789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:00:55.348460 14804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:55.348626 14789 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:55.348726 14789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "c34ffbe9a9634632bdf6fe9a98f08393"
format_stamp: "Formatted at 2026-08-06 13:00:55 on dist-test-slave-zkpd"
server_key: "8ca904da9e590f2a2b463963aa009f8b"
server_key_iv: "feaf04775fdfe5ba6dbabde3f86514d0"
server_key_version: "encryptionkey@0"
I20260806 13:00:55.348827 14789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:55.352027 14789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:55.353811 14789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:55.353960 14789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:55.357945 14789 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:46677
I20260806 13:00:55.357973 14860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:46677 every 8 connection(s)
I20260806 13:00:55.358287 14789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:55.359266 14861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:55.361435 14861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393: Bootstrap starting.
I20260806 13:00:55.361944 14861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:55.362176 14861 log.cc:824] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:55.362774 14861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393: No bootstrap required, opened a new log
I20260806 13:00:55.364229 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 14789
I20260806 13:00:55.364357 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:55.364552 24908 external_mini_cluster.cc:1428] Setting key a6832ef0b4732500016c1349802ab5a1
I20260806 13:00:55.364629 14861 raft_consensus.cc:348] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ffbe9a9634632bdf6fe9a98f08393" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46677 } }
I20260806 13:00:55.364791 14861 raft_consensus.cc:374] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:55.364817 14861 raft_consensus.cc:729] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34ffbe9a9634632bdf6fe9a98f08393, State: Initialized, Role: FOLLOWER
I20260806 13:00:55.365023 14861 consensus_queue.cc:260] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ffbe9a9634632bdf6fe9a98f08393" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46677 } }
I20260806 13:00:55.365120 14861 raft_consensus.cc:388] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:55.365193 14861 raft_consensus.cc:482] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:55.365249 14861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:55.366227 14861 raft_consensus.cc:504] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ffbe9a9634632bdf6fe9a98f08393" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46677 } }
I20260806 13:00:55.366344 14861 leader_election.cc:302] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c34ffbe9a9634632bdf6fe9a98f08393; no voters:
I20260806 13:00:55.366539 14861 leader_election.cc:288] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:55.366559 14867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:55.366741 14867 raft_consensus.cc:686] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [term 1 LEADER]: Becoming Leader. State: Replica: c34ffbe9a9634632bdf6fe9a98f08393, State: Running, Role: LEADER
I20260806 13:00:55.366824 14861 sys_catalog.cc:564] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:55.366890 14867 consensus_queue.cc:237] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ffbe9a9634632bdf6fe9a98f08393" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46677 } }
I20260806 13:00:55.366997 14866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:55.365040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50801 (local address 127.24.83.62:46677)
0806 13:00:55.365275 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:55.365284 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:55.365332 (+ 48us) server_negotiation.cc:407] Connection header received
0806 13:00:55.365445 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:55.365451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:55.365473 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:55.365596 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:55.366302 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:55.366306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:55.366306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:55.366311 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:55.366359 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:55.366369 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:55.366445 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:55.366561 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 13:00:55.366695 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":84,"threads_started":1}
I20260806 13:00:55.367223 14870 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c34ffbe9a9634632bdf6fe9a98f08393. Latest consensus state: current_term: 1 leader_uuid: "c34ffbe9a9634632bdf6fe9a98f08393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ffbe9a9634632bdf6fe9a98f08393" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46677 } } }
I20260806 13:00:55.367280 14868 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c34ffbe9a9634632bdf6fe9a98f08393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ffbe9a9634632bdf6fe9a98f08393" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 46677 } } }
I20260806 13:00:55.367408 14868 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:55.367375 14870 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:55.367743 14876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:55.367960 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:55.368232 14876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:55.370151 14876 catalog_manager.cc:1347] Generated new cluster ID: 1e20d33854d0469195f0dd7bdb5a55a6
I20260806 13:00:55.370208 14876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:55.370782 14866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:55.369599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44141 (local address 127.24.83.62:46677)
0806 13:00:55.369631 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:55.369633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:55.369838 (+ 205us) server_negotiation.cc:407] Connection header received
0806 13:00:55.369885 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:55.369885 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:55.369891 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:55.369958 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:55.370571 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:55.370573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:55.370573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:55.370578 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:55.370591 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:55.370595 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:55.370645 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:55.370696 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 13:00:55.370710 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:00:55.394838 14876 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:55.395025 14876 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:55.400197 14876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c34ffbe9a9634632bdf6fe9a98f08393: Generated new TSK 0
I20260806 13:00:55.400418 14876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:55.402463 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 14789
2026-08-06T13:00:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:55 dist-test-slave-zkpd krb5kdc[14896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:55 dist-test-slave-zkpd krb5kdc[14896](info): set up 2 sockets
Aug 06 13:00:55 dist-test-slave-zkpd krb5kdc[14896](info): commencing operation
krb5kdc: starting...
W20260806 13:00:56.436208 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:56 dist-test-slave-zkpd krb5kdc[14896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 13:00:56.550565 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:37363
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:33633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:37363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:56.635649 14909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:56.635854 14909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:56.635876 14909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:56.637059 14909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:56.637102 14909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:56.637116 14909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:56.637168 14909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:56.637197 14909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:56.638690 14909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:33633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:37363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:37363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:56.638936 14909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:56.639159 14909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:56.644191 14914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:56.644150 14917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:56.644165 14909 server_base.cc:1034] running on GCE node
W20260806 13:00:56.644148 14915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:56.644632 14909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:56.644948 14909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:56.646091 14909 hybrid_clock.cc:648] HybridClock initialized: now 1786021256646079 us; error 32 us; skew 500 ppm
Aug 06 13:00:56 dist-test-slave-zkpd krb5kdc[14896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021256, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:56.648561 14909 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:00:56.648936 14909 webserver.cc:460] Webserver started at http://127.24.83.62:44895/ using document root <none> and password file <none>
I20260806 13:00:56.649173 14909 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:56.649219 14909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:56.649346 14909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:56.650373 14909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "47d42d1bdcf043619bfcec79ba0700c4"
format_stamp: "Formatted at 2026-08-06 13:00:56 on dist-test-slave-zkpd"
server_key: "b0b3291cdb9ec9eb05daa941c8d2cbb2"
server_key_iv: "7b5da19f529763203dc1eecc5c2dd74a"
server_key_version: "encryptionkey@0"
I20260806 13:00:56.650729 14909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "47d42d1bdcf043619bfcec79ba0700c4"
format_stamp: "Formatted at 2026-08-06 13:00:56 on dist-test-slave-zkpd"
server_key: "b0b3291cdb9ec9eb05daa941c8d2cbb2"
server_key_iv: "7b5da19f529763203dc1eecc5c2dd74a"
server_key_version: "encryptionkey@0"
I20260806 13:00:56.652070 14909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:00:56.652798 14924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:56.652988 14909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:00:56.653080 14909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "47d42d1bdcf043619bfcec79ba0700c4"
format_stamp: "Formatted at 2026-08-06 13:00:56 on dist-test-slave-zkpd"
server_key: "b0b3291cdb9ec9eb05daa941c8d2cbb2"
server_key_iv: "7b5da19f529763203dc1eecc5c2dd74a"
server_key_version: "encryptionkey@0"
I20260806 13:00:56.653151 14909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:56.659325 14909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:56.661113 14909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:56.661265 14909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:56.665181 14909 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:37363
I20260806 13:00:56.665243 14978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:37363 every 8 connection(s)
I20260806 13:00:56.665584 14909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:56.666433 14979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:56.668766 14979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4: Bootstrap starting.
I20260806 13:00:56.669332 14979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:56.669615 14979 log.cc:824] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:56.670295 14979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4: No bootstrap required, opened a new log
I20260806 13:00:56.671813 14979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d42d1bdcf043619bfcec79ba0700c4" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37363 } }
I20260806 13:00:56.672006 14979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:56.672062 14979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47d42d1bdcf043619bfcec79ba0700c4, State: Initialized, Role: FOLLOWER
I20260806 13:00:56.672226 14979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d42d1bdcf043619bfcec79ba0700c4" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37363 } }
I20260806 13:00:56.672313 14979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:56.672361 14979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:56.672411 14979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:56.673203 14979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d42d1bdcf043619bfcec79ba0700c4" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37363 } }
I20260806 13:00:56.673322 14979 leader_election.cc:302] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47d42d1bdcf043619bfcec79ba0700c4; no voters:
I20260806 13:00:56.673503 14979 leader_election.cc:288] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:56.673535 14982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:56.673715 14982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [term 1 LEADER]: Becoming Leader. State: Replica: 47d42d1bdcf043619bfcec79ba0700c4, State: Running, Role: LEADER
I20260806 13:00:56.673771 14979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:56.673870 14982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d42d1bdcf043619bfcec79ba0700c4" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37363 } }
I20260806 13:00:56.674203 14983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47d42d1bdcf043619bfcec79ba0700c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d42d1bdcf043619bfcec79ba0700c4" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37363 } } }
I20260806 13:00:56.674296 14983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:56.674207 14984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47d42d1bdcf043619bfcec79ba0700c4. Latest consensus state: current_term: 1 leader_uuid: "47d42d1bdcf043619bfcec79ba0700c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d42d1bdcf043619bfcec79ba0700c4" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 37363 } } }
I20260806 13:00:56.674441 14984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:56.674561 14992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:56.675100 14992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:56.675232 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 14909
I20260806 13:00:56.675376 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:56.675560 24908 external_mini_cluster.cc:1428] Setting key 9a990336f1b4e3c12ff0836be2f8e198
I20260806 13:00:56.676952 14992 catalog_manager.cc:1347] Generated new cluster ID: 4f8feb19fbaf40808f23fd8ac9068ef2
I20260806 13:00:56.677003 14992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:56.682049 14992 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:56.682538 14992 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 13:00:56 dist-test-slave-zkpd krb5kdc[14896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:00:56.686946 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:56.676109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59119 (local address 127.24.83.62:37363)
0806 13:00:56.676287 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:56.676296 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:56.676327 (+ 31us) server_negotiation.cc:407] Connection header received
0806 13:00:56.676434 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:56.676439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:56.676480 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:56.676587 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:56.677361 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:56.678749 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:56.679850 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:56.680460 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:56.680558 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:56.683361 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:56.683365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:56.683365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:56.683375 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:56.685293 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:56.685686 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:56.685688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:56.685689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:56.685784 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:56.685998 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:56.685999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:56.685999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:56.686238 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:56.686397 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:56.686505 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 13:00:56.686730 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260806 13:00:56.688093 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:56.695539 14992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47d42d1bdcf043619bfcec79ba0700c4: Generated new TSK 0
I20260806 13:00:56.695761 14992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:56.696589 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:56.689519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38245 (local address 127.24.83.62:37363)
0806 13:00:56.689561 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:56.689563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:56.689656 (+ 93us) server_negotiation.cc:407] Connection header received
0806 13:00:56.689732 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:56.689733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:56.689740 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:56.689766 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:56.690384 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:56.691088 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:56.692110 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:56.692415 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:56.692449 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:56.693797 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:56.693799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:56.693799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:00:56.693805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:56.695309 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:56.695694 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:56.695696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:56.695697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:56.695763 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:00:56.696077 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:00:56.696078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:00:56.696078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:00:56.696206 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:56.696305 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:56.696422 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 13:00:56.696502 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:00:56.698236 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 14909
2026-08-06T13:00:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:56 dist-test-slave-zkpd krb5kdc[15011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:56 dist-test-slave-zkpd krb5kdc[15011](info): set up 2 sockets
Aug 06 13:00:56 dist-test-slave-zkpd krb5kdc[15011](info): commencing operation
krb5kdc: starting...
W20260806 13:00:57.955875 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:58 dist-test-slave-zkpd krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 13:00:58.067023 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45815
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:45815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:58.148598 15025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:58.148854 15025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:58.148897 15025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:58.150068 15025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:58.150142 15025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:58.150167 15025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:58.150208 15025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:58.150235 15025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:58.151748 15025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:45815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:45815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:58.152066 15025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:58.152312 15025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:58.157685 15034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:58.157708 15025 server_base.cc:1034] running on GCE node
W20260806 13:00:58.157676 15032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:58.157655 15031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:58.158133 15025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:58.158460 15025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:58.159632 15025 hybrid_clock.cc:648] HybridClock initialized: now 1786021258159620 us; error 29 us; skew 500 ppm
Aug 06 13:00:58 dist-test-slave-zkpd krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021258, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:58.162175 15025 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:00:58.162607 15025 webserver.cc:460] Webserver started at http://127.24.83.62:35219/ using document root <none> and password file <none>
I20260806 13:00:58.162844 15025 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:58.162915 15025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:58.163067 15025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:58.164264 15025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "3252d4dea2354f949184d0482ba8d99a"
format_stamp: "Formatted at 2026-08-06 13:00:58 on dist-test-slave-zkpd"
server_key: "32e447848a269ed40b1c11bec45d2c15"
server_key_iv: "b29b0ff060a271744682425d956f5052"
server_key_version: "encryptionkey@0"
I20260806 13:00:58.164651 15025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "3252d4dea2354f949184d0482ba8d99a"
format_stamp: "Formatted at 2026-08-06 13:00:58 on dist-test-slave-zkpd"
server_key: "32e447848a269ed40b1c11bec45d2c15"
server_key_iv: "b29b0ff060a271744682425d956f5052"
server_key_version: "encryptionkey@0"
I20260806 13:00:58.166086 15025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:00:58.166903 15041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:58.167104 15025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:00:58.167227 15025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "3252d4dea2354f949184d0482ba8d99a"
format_stamp: "Formatted at 2026-08-06 13:00:58 on dist-test-slave-zkpd"
server_key: "32e447848a269ed40b1c11bec45d2c15"
server_key_iv: "b29b0ff060a271744682425d956f5052"
server_key_version: "encryptionkey@0"
I20260806 13:00:58.167326 15025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:58.190261 15025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:58.192302 15025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:58.192474 15025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:58.196470 15025 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:45815
I20260806 13:00:58.196517 15094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:45815 every 8 connection(s)
I20260806 13:00:58.196903 15025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:58.197831 15095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:58.200397 15095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a: Bootstrap starting.
I20260806 13:00:58.201021 15095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:58.201346 15095 log.cc:824] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:58.201694 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 15025
I20260806 13:00:58.201830 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:58.202016 24908 external_mini_cluster.cc:1428] Setting key 18ce6daea00cb4fe21363b94ee77063f
I20260806 13:00:58.202231 15095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a: No bootstrap required, opened a new log
I20260806 13:00:58.205451 15095 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3252d4dea2354f949184d0482ba8d99a" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45815 } }
I20260806 13:00:58.205631 15095 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:58.205663 15095 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3252d4dea2354f949184d0482ba8d99a, State: Initialized, Role: FOLLOWER
I20260806 13:00:58.205812 15095 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3252d4dea2354f949184d0482ba8d99a" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45815 } }
I20260806 13:00:58.205941 15095 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:58.205994 15095 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:58.206054 15095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:58.206676 15095 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3252d4dea2354f949184d0482ba8d99a" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45815 } }
I20260806 13:00:58.206799 15095 leader_election.cc:302] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3252d4dea2354f949184d0482ba8d99a; no voters:
I20260806 13:00:58.206969 15095 leader_election.cc:288] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:58.207017 15100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:58.207180 15100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [term 1 LEADER]: Becoming Leader. State: Replica: 3252d4dea2354f949184d0482ba8d99a, State: Running, Role: LEADER
I20260806 13:00:58.207284 15095 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:58.207332 15100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3252d4dea2354f949184d0482ba8d99a" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45815 } }
I20260806 13:00:58.207737 15100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3252d4dea2354f949184d0482ba8d99a. Latest consensus state: current_term: 1 leader_uuid: "3252d4dea2354f949184d0482ba8d99a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3252d4dea2354f949184d0482ba8d99a" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45815 } } }
I20260806 13:00:58.207815 15102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3252d4dea2354f949184d0482ba8d99a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3252d4dea2354f949184d0482ba8d99a" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 45815 } } }
I20260806 13:00:58.207864 15100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:58.207930 15102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:58.208428 15104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:58.208974 15104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:58.209641 15099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:58.202555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40045 (local address 127.24.83.62:45815)
0806 13:00:58.202777 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:58.202787 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:58.202825 (+ 38us) server_negotiation.cc:407] Connection header received
0806 13:00:58.203715 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:58.203726 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:58.203795 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:58.203978 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:58.204207 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:58.205813 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:58.206718 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:58.207404 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:58.207543 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:58.208293 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:58.208298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:58.208299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:58.208540 (+ 241us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:58.208589 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:58.208609 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:58.208680 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:58.209033 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 13:00:58.209105 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":100,"threads_started":1}
I20260806 13:00:58.210633 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:58.213434 15104 catalog_manager.cc:1347] Generated new cluster ID: d9660d1a25ac4430a77e939dd4be1491
I20260806 13:00:58.213495 15104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:58.215484 15099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:58.212165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43839 (local address 127.24.83.62:45815)
0806 13:00:58.212207 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:58.212209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:58.212348 (+ 139us) server_negotiation.cc:407] Connection header received
0806 13:00:58.212380 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:58.212382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:58.212394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:58.212459 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:58.212649 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:58.213279 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:58.214169 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:00:58.214405 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:00:58.214499 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:00:58.215228 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:58.215230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:58.215230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:58.215235 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:58.215249 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:58.215253 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:58.215311 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:58.215380 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 13:00:58.215400 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:00:58.222934 15104 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:58.223632 15104 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:58.231114 15104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3252d4dea2354f949184d0482ba8d99a: Generated new TSK 0
I20260806 13:00:58.231297 15104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:58.238242 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 15025
2026-08-06T13:00:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:58 dist-test-slave-zkpd krb5kdc[15129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:58 dist-test-slave-zkpd krb5kdc[15129](info): set up 2 sockets
Aug 06 13:00:58 dist-test-slave-zkpd krb5kdc[15129](info): commencing operation
krb5kdc: starting...
W20260806 13:00:59.284065 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:00:59 dist-test-slave-zkpd krb5kdc[15129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:00:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:00:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 13:00:59.402688 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:42819
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:34371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:42819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:00:59.485419 15142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:00:59.485607 15142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:00:59.485628 15142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:00:59.486696 15142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:00:59.486729 15142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:00:59.486743 15142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:00:59.486756 15142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:00:59.486768 15142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:00:59.488222 15142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:34371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:42819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:42819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:00:59.488412 15142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:00:59.488570 15142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:00:59.493247 15148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:59.493256 15147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:00:59.493259 15150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:00:59.493423 15142 server_base.cc:1034] running on GCE node
I20260806 13:00:59.493779 15142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:00:59.494091 15142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:00:59.495232 15142 hybrid_clock.cc:648] HybridClock initialized: now 1786021259495211 us; error 36 us; skew 500 ppm
Aug 06 13:00:59 dist-test-slave-zkpd krb5kdc[15129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021259, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:00:59.497730 15142 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:00:59.498137 15142 webserver.cc:460] Webserver started at http://127.24.83.62:43061/ using document root <none> and password file <none>
I20260806 13:00:59.498389 15142 fs_manager.cc:359] Metadata directory not provided
I20260806 13:00:59.498431 15142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:00:59.498588 15142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:00:59.499569 15142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "e1e9f14d95aa43c2ba596c305c0a54b6"
format_stamp: "Formatted at 2026-08-06 13:00:59 on dist-test-slave-zkpd"
server_key: "bac842b5ccae0307392b755c38fe5f10"
server_key_iv: "215d5126be097995f01fc55569cc3367"
server_key_version: "encryptionkey@0"
I20260806 13:00:59.500000 15142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "e1e9f14d95aa43c2ba596c305c0a54b6"
format_stamp: "Formatted at 2026-08-06 13:00:59 on dist-test-slave-zkpd"
server_key: "bac842b5ccae0307392b755c38fe5f10"
server_key_iv: "215d5126be097995f01fc55569cc3367"
server_key_version: "encryptionkey@0"
I20260806 13:00:59.501273 15142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:00:59.501965 15157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:00:59.502139 15142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:00:59.502235 15142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "e1e9f14d95aa43c2ba596c305c0a54b6"
format_stamp: "Formatted at 2026-08-06 13:00:59 on dist-test-slave-zkpd"
server_key: "bac842b5ccae0307392b755c38fe5f10"
server_key_iv: "215d5126be097995f01fc55569cc3367"
server_key_version: "encryptionkey@0"
I20260806 13:00:59.502319 15142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:00:59.505522 15142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:00:59.507723 15142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:00:59.507939 15142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:00:59.511638 15142 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:42819
I20260806 13:00:59.511696 15213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:42819 every 8 connection(s)
I20260806 13:00:59.512080 15142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:00:59.513070 15214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:00:59.515215 15214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6: Bootstrap starting.
I20260806 13:00:59.515774 15214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6: Neither blocks nor log segments found. Creating new log.
I20260806 13:00:59.516103 15214 log.cc:824] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6: Log is configured to *not* fsync() on all Append() calls
I20260806 13:00:59.516782 15214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6: No bootstrap required, opened a new log
I20260806 13:00:59.516847 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 15142
I20260806 13:00:59.516957 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:00:59.517153 24908 external_mini_cluster.cc:1428] Setting key 90e2689fe684292d13015f7612d4753a
I20260806 13:00:59.518679 15214 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f14d95aa43c2ba596c305c0a54b6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42819 } }
I20260806 13:00:59.518851 15214 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:00:59.518965 15214 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1e9f14d95aa43c2ba596c305c0a54b6, State: Initialized, Role: FOLLOWER
I20260806 13:00:59.519119 15214 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f14d95aa43c2ba596c305c0a54b6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42819 } }
I20260806 13:00:59.519220 15214 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:00:59.519258 15214 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:00:59.519307 15214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:00:59.520028 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:59.517652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34273 (local address 127.24.83.62:42819)
0806 13:00:59.517821 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:59.517829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:59.517861 (+ 32us) server_negotiation.cc:407] Connection header received
0806 13:00:59.518663 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:59.518669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:59.518694 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:59.518805 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:59.519515 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:59.519519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:59.519519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:59.519524 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:59.519573 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:59.519589 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:59.519667 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:59.519732 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 13:00:59.519778 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260806 13:00:59.520444 15214 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f14d95aa43c2ba596c305c0a54b6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42819 } }
I20260806 13:00:59.520546 15214 leader_election.cc:302] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1e9f14d95aa43c2ba596c305c0a54b6; no voters:
I20260806 13:00:59.520691 15214 leader_election.cc:288] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:00:59.520714 15219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:00:59.520864 15219 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [term 1 LEADER]: Becoming Leader. State: Replica: e1e9f14d95aa43c2ba596c305c0a54b6, State: Running, Role: LEADER
I20260806 13:00:59.520944 15214 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:00:59.520962 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:00:59.521005 15219 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f14d95aa43c2ba596c305c0a54b6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42819 } }
I20260806 13:00:59.521296 15219 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1e9f14d95aa43c2ba596c305c0a54b6. Latest consensus state: current_term: 1 leader_uuid: "e1e9f14d95aa43c2ba596c305c0a54b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f14d95aa43c2ba596c305c0a54b6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42819 } } }
I20260806 13:00:59.521368 15219 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:59.521307 15221 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1e9f14d95aa43c2ba596c305c0a54b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f14d95aa43c2ba596c305c0a54b6" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42819 } } }
I20260806 13:00:59.521425 15221 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6 [sys.catalog]: This master's current role is: LEADER
I20260806 13:00:59.521584 15224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:00:59.522099 15224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:00:59.524143 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:00:59.522753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60649 (local address 127.24.83.62:42819)
0806 13:00:59.522794 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:00:59.522796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:00:59.523146 (+ 350us) server_negotiation.cc:407] Connection header received
0806 13:00:59.523214 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:00:59.523214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:00:59.523219 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:00:59.523311 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:00:59.523926 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:00:59.523927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:00:59.523928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:00:59.523932 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:00:59.523946 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:00:59.523951 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:00:59.524019 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:00:59.524073 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 13:00:59.524087 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:00:59.524230 15224 catalog_manager.cc:1347] Generated new cluster ID: 838ec7221c02455b89dd9d63be486850
I20260806 13:00:59.524273 15224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:00:59.530540 15224 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:00:59.530686 15224 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:00:59.544855 15224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1e9f14d95aa43c2ba596c305c0a54b6: Generated new TSK 0
I20260806 13:00:59.545037 15224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:00:59.547319 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 15142
2026-08-06T13:00:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1309 ms)
[----------] 18 tests from AuthTokenIssuingTest (24020 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:00:59 dist-test-slave-zkpd krb5kdc[15245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:00:59 dist-test-slave-zkpd krb5kdc[15245](info): set up 2 sockets
Aug 06 13:00:59 dist-test-slave-zkpd krb5kdc[15245](info): commencing operation
krb5kdc: starting...
W20260806 13:01:00.596257 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:01:00 dist-test-slave-zkpd krb5kdc[15245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:01:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:01:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 13:01:00.710557 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:42255
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:42255,127.24.83.61:45423,127.24.83.60:39247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:01:00.790320 15258 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:01:00.790570 15258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:01:00.790601 15258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:01:00.790642 15258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:01:00.791738 15258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:01:00.791805 15258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:01:00.791829 15258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:01:00.791857 15258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:01:00.791905 15258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:01:00.793423 15258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:42255,127.24.83.61:45423,127.24.83.60:39247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:42255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:01:00.793725 15258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:01:00.793955 15258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 13:01:00.798740 15258 server_base.cc:1034] running on GCE node
W20260806 13:01:00.798712 15266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:01:00.798709 15264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:01:00.798712 15263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:01:00.799327 15258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:01:00.799633 15258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:01:00.800787 15258 hybrid_clock.cc:648] HybridClock initialized: now 1786021260800771 us; error 27 us; skew 500 ppm
Aug 06 13:01:00 dist-test-slave-zkpd krb5kdc[15245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:01:00.803166 15258 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:01:00.803527 15258 webserver.cc:460] Webserver started at http://127.24.83.62:40149/ using document root <none> and password file <none>
I20260806 13:01:00.803709 15258 fs_manager.cc:359] Metadata directory not provided
I20260806 13:01:00.803747 15258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:01:00.803831 15258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:01:00.804894 15258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "e3ead5df328d4f66bba829b1c08dacd8"
format_stamp: "Formatted at 2026-08-06 13:01:00 on dist-test-slave-zkpd"
server_key: "3642b4347555f9762bcae9189cd0d413"
server_key_iv: "bf857f8bde729ade93e7a67c73618899"
server_key_version: "encryptionkey@0"
I20260806 13:01:00.805190 15258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "e3ead5df328d4f66bba829b1c08dacd8"
format_stamp: "Formatted at 2026-08-06 13:01:00 on dist-test-slave-zkpd"
server_key: "3642b4347555f9762bcae9189cd0d413"
server_key_iv: "bf857f8bde729ade93e7a67c73618899"
server_key_version: "encryptionkey@0"
I20260806 13:01:00.806459 15258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:01:00.807097 15273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:01:00.807263 15258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:01:00.807356 15258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "e3ead5df328d4f66bba829b1c08dacd8"
format_stamp: "Formatted at 2026-08-06 13:01:00 on dist-test-slave-zkpd"
server_key: "3642b4347555f9762bcae9189cd0d413"
server_key_iv: "bf857f8bde729ade93e7a67c73618899"
server_key_version: "encryptionkey@0"
I20260806 13:01:00.807423 15258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:01:00.814391 15258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:01:00.816177 15258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:01:00.816327 15258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:01:00.820158 15258 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:42255
I20260806 13:01:00.820204 15329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:42255 every 8 connection(s)
I20260806 13:01:00.820538 15258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:01:00.821489 15330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:01:00.822911 15330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:01:00.824827 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 15258
I20260806 13:01:00.824965 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:01:00.825172 24908 external_mini_cluster.cc:1428] Setting key 1c689e1e5f7fd35c01e0c332b6fafe39
Aug 06 13:01:00 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:01:00.833034 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.823562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:42255 (local address 127.0.0.1:33808)
0806 13:01:00.823738 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:00.823773 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:00.824249 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:00.824496 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:00.824498 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:00.824816 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:00.825045 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:00.825052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.826093 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.826094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:00.827025 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:00.827028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.827716 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.827717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:00.827768 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:00.828294 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:00.828302 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:00.829619 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:00.831632 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:00.831639 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:00.831640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:00.831824 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:00.832130 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:00.832130 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:00.832131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:00.832207 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:00.832644 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:00.832648 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:00.832698 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 13:01:00.832763 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 13:01:00.832882 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260806 13:01:00.833118 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.823603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33808 (local address 127.24.83.62:42255)
0806 13:01:00.823742 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:00.823749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:00.823868 (+ 119us) server_negotiation.cc:407] Connection header received
0806 13:01:00.824372 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:00.824374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:00.824390 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:00.824467 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:00.825144 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.825992 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.827108 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.827613 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.827723 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:00.829704 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:00.829713 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:00.829713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:00.829721 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:00.831551 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:00.831949 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:00.831952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:00.831953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:00.832054 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:00.832279 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:00.832280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:00.832280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:00.832563 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:00.832655 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:00.832838 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 13:01:00.832971 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.24.83.61@KRBTEST.COM; defaulting to no policy
I20260806 13:01:00.833748 15330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:01:00.834065 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.833956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:45423 (local address 127.0.0.1:42248)
0806 13:01:00.834004 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:00.834046 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:45423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 13:01:00.834224 15276 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.83.61:45423: connect: Connection refused (error 111)
I20260806 13:01:00.834635 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.834538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:45423 (local address 127.0.0.1:42264)
0806 13:01:00.834583 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:00.834613 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:45423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 13:01:00.834733 15330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.61:45423: Network error: Client connection negotiation failed: client connection to 127.24.83.61:45423: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab.
Entry for principal kudu/127.24.83.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab.
I20260806 13:01:00.848675 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.61:45423
--webserver_interface=127.24.83.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab
--principal=kudu/127.24.83.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:42255,127.24.83.61:45423,127.24.83.60:39247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:01:00.884007 15330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } attempt: 1
I20260806 13:01:00.884493 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.884357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:45423 (local address 127.0.0.1:42268)
0806 13:01:00.884429 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:00.884464 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:45423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 13:01:00.884840 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.884758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:45423 (local address 127.0.0.1:42274)
0806 13:01:00.884784 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:00.884810 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:45423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 13:01:00.884933 15330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.61:45423: Network error: Client connection negotiation failed: client connection to 127.24.83.61:45423: connect: Connection refused (error 111)
W20260806 13:01:00.929258 15338 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:01:00.929498 15338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:01:00.929528 15338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:01:00.929569 15338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:01:00.930683 15338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:01:00.930750 15338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:01:00.930776 15338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:01:00.930806 15338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:01:00.930830 15338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:01:00.932346 15338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:42255,127.24.83.61:45423,127.24.83.60:39247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab
--principal=kudu/127.24.83.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.61:45423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.83.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:01:00.932622 15338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:01:00.932849 15338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:01:00.937537 15346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:01:00.937695 15338 server_base.cc:1034] running on GCE node
W20260806 13:01:00.937582 15344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:01:00.937561 15343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:01:00.938055 15338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:01:00.938339 15338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:01:00.939487 15338 hybrid_clock.cc:648] HybridClock initialized: now 1786021260939468 us; error 35 us; skew 500 ppm
Aug 06 13:01:00 dist-test-slave-zkpd krb5kdc[15245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:01:00.941900 15338 init.cc:377] Logged in from keytab as kudu/127.24.83.61@KRBTEST.COM (short username kudu)
I20260806 13:01:00.942353 15338 webserver.cc:460] Webserver started at http://127.24.83.61:36657/ using document root <none> and password file <none>
I20260806 13:01:00.942591 15338 fs_manager.cc:359] Metadata directory not provided
I20260806 13:01:00.942660 15338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:01:00.942811 15338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:01:00.943811 15338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/data/instance:
uuid: "bb3410c2a2d24e4894a17cbde3335fdd"
format_stamp: "Formatted at 2026-08-06 13:01:00 on dist-test-slave-zkpd"
server_key: "75ebcb6997ee437a352dda941660593b"
server_key_iv: "e950a86fdb05ea2dfaa9a6e9ed7499b0"
server_key_version: "encryptionkey@0"
I20260806 13:01:00.944247 15338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/wal/instance:
uuid: "bb3410c2a2d24e4894a17cbde3335fdd"
format_stamp: "Formatted at 2026-08-06 13:01:00 on dist-test-slave-zkpd"
server_key: "75ebcb6997ee437a352dda941660593b"
server_key_iv: "e950a86fdb05ea2dfaa9a6e9ed7499b0"
server_key_version: "encryptionkey@0"
I20260806 13:01:00.945529 15338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:01:00.946280 15354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:01:00.946475 15338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:01:00.946578 15338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/wal
uuid: "bb3410c2a2d24e4894a17cbde3335fdd"
format_stamp: "Formatted at 2026-08-06 13:01:00 on dist-test-slave-zkpd"
server_key: "75ebcb6997ee437a352dda941660593b"
server_key_iv: "e950a86fdb05ea2dfaa9a6e9ed7499b0"
server_key_version: "encryptionkey@0"
I20260806 13:01:00.946652 15338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:01:00.953083 15330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } attempt: 2
I20260806 13:01:00.953471 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.953343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:45423 (local address 127.0.0.1:42276)
0806 13:01:00.953407 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:00.953443 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:45423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 13:01:00.953815 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.953722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:45423 (local address 127.0.0.1:42286)
0806 13:01:00.953766 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:00.953790 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:45423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 13:01:00.953898 15330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.61:45423: Network error: Client connection negotiation failed: client connection to 127.24.83.61:45423: connect: Connection refused (error 111)
I20260806 13:01:00.961262 15338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:01:00.963019 15338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:01:00.963169 15338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:01:00.967336 15338 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.61:45423
I20260806 13:01:00.967368 15406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.61:45423 every 8 connection(s)
I20260806 13:01:00.967700 15338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
I20260806 13:01:00.968647 15407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:01:00.970423 15407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:01:00.972841 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 15338
I20260806 13:01:00.972966 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-1/wal/instance
I20260806 13:01:00.973167 24908 external_mini_cluster.cc:1428] Setting key 5fc1e143bdc469501f07f0be3c4a7311
Aug 06 13:01:00 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:01:00.979871 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.971106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33822 (local address 127.24.83.62:42255)
0806 13:01:00.971179 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:00.971181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:00.971336 (+ 155us) server_negotiation.cc:407] Connection header received
0806 13:01:00.971772 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:00.971773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:00.971782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:00.971842 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:00.972442 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.972916 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.974401 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.974638 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.974733 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:00.976898 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:00.976900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:00.976900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:00.976909 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:00.978541 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:00.978908 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:00.978910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:00.978910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:00.978978 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:00.979211 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:00.979213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:00.979213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:00.979362 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:00.979446 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:00.979672 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 13:01:00.979770 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 13:01:00.979874 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.971056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:42255 (local address 127.0.0.1:33822)
0806 13:01:00.971214 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:00.971254 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:00.971690 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:00.971947 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:00.971955 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:00.972179 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:00.972359 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:00.972363 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.973027 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.973028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:00.974303 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:00.974306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.974755 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.974757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:00.974890 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:00.975369 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:00.975385 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:00.976741 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:00.978624 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:00.978630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:00.978632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:00.978833 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:00.979052 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:00.979053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:00.979053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:00.979144 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:00.979434 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:00.979438 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:00.979547 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 13:01:00.979625 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 13:01:00.979731 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260806 13:01:00.980593 15407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.83.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.60@KRBTEST.COM" created.
Aug 06 13:01:00 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
I20260806 13:01:00.988052 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.980861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:45423 (local address 127.0.0.1:42300)
0806 13:01:00.980913 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:00.980923 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:00.980963 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:00.981219 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:00.981220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:00.981355 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:00.981413 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:00.981414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.982303 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.982304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:00.982851 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:00.982853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.983324 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.983325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:00.983397 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:00.983849 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:00.983854 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:00.985139 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:00.986952 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:00.986953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:00.986953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:00.987112 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:00.987332 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:00.987333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:00.987333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:00.987394 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:00.987674 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:00.987674 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:00.987711 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 13:01:00.987813 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 13:01:00.987938 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 13:01:00.988121 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.980937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42300 (local address 127.24.83.61:45423)
0806 13:01:00.981095 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:00.981099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:00.981113 (+ 14us) server_negotiation.cc:407] Connection header received
0806 13:01:00.981128 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:00.981130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:00.981140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:00.981220 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:00.981484 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.982215 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.982944 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:00.983249 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:00.983327 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:00.985219 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:00.985227 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:00.985227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:00.985235 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:00.986841 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:00.987201 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:00.987204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:00.987205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:00.987258 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:00.987460 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:00.987461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:00.987461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:00.987604 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:00.987694 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:00.987847 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 13:01:00.987991 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260806 13:01:00.988656 15407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:01:00.988991 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.988895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45424)
0806 13:01:00.988932 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:00.988971 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 13:01:00.989147 15356 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
I20260806 13:01:00.989524 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:00.989462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45432)
0806 13:01:00.989487 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:00.989509 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 13:01:00.989655 15407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:39247: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab.
Entry for principal kudu/127.24.83.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab.
I20260806 13:01:00.998400 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.60:39247
--webserver_interface=127.24.83.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab
--principal=kudu/127.24.83.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:42255,127.24.83.61:45423,127.24.83.60:39247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:01:01.038908 15407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } attempt: 1
I20260806 13:01:01.039439 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.039294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45446)
0806 13:01:01.039359 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.039403 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 13:01:01.039819 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.039727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45450)
0806 13:01:01.039774 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.039800 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 13:01:01.039963 15407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:39247: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
I20260806 13:01:01.045038 15330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } attempt: 3
Aug 06 13:01:01 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
I20260806 13:01:01.052563 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.045460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.24.83.61:45423)
0806 13:01:01.045505 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.045507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.045518 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:01:01.045594 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.045595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.045608 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.045679 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.046033 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.046546 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.047272 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.047499 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.047587 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.049784 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.049785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.049786 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.049794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.051367 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.051776 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.051776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.051777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.051831 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.052092 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.052094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.052094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.052209 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.052270 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.052438 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.052504 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:01:01.052605 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.045405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:45423 (local address 127.0.0.1:42312)
0806 13:01:01.045461 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.045472 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:01.045527 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:01.045686 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:01.045687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:01.045865 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:01.045944 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.045946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.046640 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.046641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.047178 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.047180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.047588 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.047589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.047672 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.048173 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:01.048185 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:01.049656 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:01.051458 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.051459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.051460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.051678 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.051937 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.051938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.051938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.052009 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.052280 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:01.052281 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:01.052330 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 13:01:01.052409 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 13:01:01.052507 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 13:01:01.053337 15330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:01:01.053691 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.053576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45466)
0806 13:01:01.053623 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.053655 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 13:01:01.054031 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.053945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45478)
0806 13:01:01.053988 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.054014 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 13:01:01.054158 15330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:39247: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
W20260806 13:01:01.082041 15416 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:01:01.082264 15416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:01:01.082296 15416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:01:01.082327 15416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:01:01.083487 15416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:01:01.083525 15416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:01:01.083539 15416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:01:01.083573 15416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:01:01.083601 15416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:01:01.085178 15416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:42255,127.24.83.61:45423,127.24.83.60:39247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab
--principal=kudu/127.24.83.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.60:39247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.83.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:01:01.085326 15330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } attempt: 1
I20260806 13:01:01.085436 15416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:01:01.085682 15416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 13:01:01.085764 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.085621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45486)
0806 13:01:01.085681 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.085731 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 13:01:01.086153 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.086052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45500)
0806 13:01:01.086103 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.086131 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 13:01:01.086285 15330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:39247: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
W20260806 13:01:01.090353 15424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:01:01.090400 15422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:01:01.090404 15421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:01:01.090734 15416 server_base.cc:1034] running on GCE node
I20260806 13:01:01.090967 15416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:01:01.091235 15416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:01:01.092394 15416 hybrid_clock.cc:648] HybridClock initialized: now 1786021261092386 us; error 34 us; skew 500 ppm
Aug 06 13:01:01 dist-test-slave-zkpd krb5kdc[15245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021261, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:01:01.094683 15416 init.cc:377] Logged in from keytab as kudu/127.24.83.60@KRBTEST.COM (short username kudu)
I20260806 13:01:01.095090 15416 webserver.cc:460] Webserver started at http://127.24.83.60:37847/ using document root <none> and password file <none>
I20260806 13:01:01.095304 15416 fs_manager.cc:359] Metadata directory not provided
I20260806 13:01:01.095369 15416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:01:01.095513 15416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:01:01.096580 15416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/data/instance:
uuid: "8dce7853c4144d0498bd74a942be969b"
format_stamp: "Formatted at 2026-08-06 13:01:01 on dist-test-slave-zkpd"
server_key: "032dca76713be940c2c3091c4a39bdfd"
server_key_iv: "67ab8e743fbd0237657b3a7d3ecd26b7"
server_key_version: "encryptionkey@0"
I20260806 13:01:01.096935 15416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/wal/instance:
uuid: "8dce7853c4144d0498bd74a942be969b"
format_stamp: "Formatted at 2026-08-06 13:01:01 on dist-test-slave-zkpd"
server_key: "032dca76713be940c2c3091c4a39bdfd"
server_key_iv: "67ab8e743fbd0237657b3a7d3ecd26b7"
server_key_version: "encryptionkey@0"
I20260806 13:01:01.098230 15416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:01:01.098937 15431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:01:01.099133 15416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:01:01.099229 15416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/wal
uuid: "8dce7853c4144d0498bd74a942be969b"
format_stamp: "Formatted at 2026-08-06 13:01:01 on dist-test-slave-zkpd"
server_key: "032dca76713be940c2c3091c4a39bdfd"
server_key_iv: "67ab8e743fbd0237657b3a7d3ecd26b7"
server_key_version: "encryptionkey@0"
I20260806 13:01:01.099306 15416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:01:01.108137 15407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } attempt: 2
I20260806 13:01:01.108556 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.108423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45506)
0806 13:01:01.108482 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.108521 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 13:01:01.108875 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.108797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45520)
0806 13:01:01.108834 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.108857 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 13:01:01.108982 15407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:39247: Network error: Client connection negotiation failed: client connection to 127.24.83.60:39247: connect: Connection refused (error 111)
I20260806 13:01:01.120716 15416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:01:01.122534 15416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:01:01.122696 15416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:01:01.126550 15416 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.60:39247
I20260806 13:01:01.126629 15485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.60:39247 every 8 connection(s)
I20260806 13:01:01.126962 15416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
I20260806 13:01:01.127987 15486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:01:01.129524 15486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:01:01.132917 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 15416
I20260806 13:01:01.133008 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786020991335182-24908-0/minicluster-data/master-2/wal/instance
I20260806 13:01:01.133225 24908 external_mini_cluster.cc:1428] Setting key 2907e05c5b11c36ae8e92336601397d7
Aug 06 13:01:01 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021261, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
Aug 06 13:01:01 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
I20260806 13:01:01.139387 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.130184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33826 (local address 127.24.83.62:42255)
0806 13:01:01.130231 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.130233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.130434 (+ 201us) server_negotiation.cc:407] Connection header received
0806 13:01:01.130958 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.130960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.130974 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.131022 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.131644 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.132184 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.133287 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.133491 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.133582 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.135797 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.135799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.135799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.135806 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.137994 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.138416 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.138418 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.138418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.138480 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.138743 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.138744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.138745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.138887 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.138955 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.139224 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.139308 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:01:01.139591 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.130140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:42255 (local address 127.0.0.1:33826)
0806 13:01:01.130314 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.130352 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:01.130848 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:01.131102 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:01.131108 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:01.131353 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:01.131542 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.131546 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.132310 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.132311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.133200 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.133202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.133625 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.133626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.133821 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.134314 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:01.134323 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:01.135732 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:01.138097 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.138105 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.138108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.138331 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.138550 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.138551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.138551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.138667 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.138961 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:01.138968 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:01.139081 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 13:01:01.139173 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 13:01:01.139326 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260806 13:01:01.140271 15486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:01:01.142552 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.133603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45528 (local address 127.24.83.60:39247)
0806 13:01:01.133792 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.133796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.133818 (+ 22us) server_negotiation.cc:407] Connection header received
0806 13:01:01.133867 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.133870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.133890 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.133976 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.134835 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.135641 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.136616 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.136921 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.137002 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.139290 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.139312 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.139312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.139319 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.141446 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.141768 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.141771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.141771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.141837 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.142043 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.142044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.142044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.142173 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.142267 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.142399 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.142462 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
Aug 06 13:01:01 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021261, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
I20260806 13:01:01.147629 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.140571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:45423 (local address 127.0.0.1:42326)
0806 13:01:01.140619 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.140631 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:01.140673 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:01.140842 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:01.140842 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:01.140954 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:01.140983 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.140984 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.141813 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.141814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.142521 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.142523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.142985 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.142986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.143052 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.143450 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:01.143455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:01.144938 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:01.146599 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.146600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.146600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.146844 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.147036 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.147037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.147037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.147106 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.147419 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:01.147420 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:01.147453 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 13:01:01.147506 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 13:01:01.147572 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 13:01:01.147720 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.140643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42326 (local address 127.24.83.61:45423)
0806 13:01:01.140711 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.140713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.140723 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:01:01.140751 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.140752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.140763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.140841 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.141054 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.141707 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.142595 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.142894 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.142984 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.145023 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.145024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.145024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.145031 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.146517 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.146927 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.146927 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.146928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.146975 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.147196 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.147197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.147198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.147347 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.147416 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.147555 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.147628 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 13:01:01 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:01:01.148221 15486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:01:01.151100 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.143434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.24.83.62:42255)
0806 13:01:01.143474 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.143476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.143484 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:01:01.143503 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.143504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.143512 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.143566 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.144187 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.144849 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.145544 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.145748 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.145830 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.148269 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.148270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.148271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.148277 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.149953 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.150287 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.150288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.150288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.150348 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.150574 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.150576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.150576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.150683 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.150860 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.150898 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.150987 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 06 13:01:01 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021261, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
I20260806 13:01:01.155434 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.148480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45542)
0806 13:01:01.148576 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.148585 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:01.148628 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:01.148755 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:01.148755 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:01.148923 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:01.148958 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.148960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.149678 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.149679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.150354 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.150357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.150822 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.150823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.150882 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.151415 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:01.151419 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:01.152725 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:01.154339 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.154340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.154340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.154498 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.154699 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.154700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.154701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.154770 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.155075 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:01.155076 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:01.155123 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 13:01:01.155202 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 13:01:01.155303 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 13:01:01.155441 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.148542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45542 (local address 127.24.83.60:39247)
0806 13:01:01.148576 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.148578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.148620 (+ 42us) server_negotiation.cc:407] Connection header received
0806 13:01:01.148690 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.148691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.148701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.148753 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.149041 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.149602 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.150440 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.150740 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.150801 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.152800 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.152801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.152801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.152808 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.154250 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.154564 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.154565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.154565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.154611 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.154857 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.154858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.154858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.154998 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.155066 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.155234 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.155361 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":16}
Aug 06 13:01:01 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
I20260806 13:01:01.156991 15486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b: Bootstrap starting.
I20260806 13:01:01.157989 15486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b: Neither blocks nor log segments found. Creating new log.
I20260806 13:01:01.158314 15486 log.cc:824] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b: Log is configured to *not* fsync() on all Append() calls
I20260806 13:01:01.159232 15486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b: No bootstrap required, opened a new log
I20260806 13:01:01.159695 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.151740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.24.83.61:45423)
0806 13:01:01.151784 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.151786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.151794 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:01:01.151814 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.151814 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.151822 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.151884 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.152488 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.153055 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.153928 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.154263 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.154327 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.157128 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.157130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.157130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.157137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.158619 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.158982 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.158983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.158983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.159033 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.159234 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.159235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.159235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.159377 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.159496 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.159575 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.159636 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:01:01.161430 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:01:01.161438 15330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } attempt: 2
I20260806 13:01:01.161434 15486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } }
I20260806 13:01:01.161587 15486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dce7853c4144d0498bd74a942be969b, State: Initialized, Role: FOLLOWER
I20260806 13:01:01.161685 15486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } }
I20260806 13:01:01.161988 15492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } } }
I20260806 13:01:01.162046 15486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:01:01.162091 15492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:01.162456 15302 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e3ead5df328d4f66bba829b1c08dacd8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33832
W20260806 13:01:01.164275 15504 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:01:01.164330 15504 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:01:01.164085 15416 master_runner.cc:186] Error getting master registration for 127.24.83.62:42255: OK, instance_id { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" instance_seqno: 1786021260819065 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 13:01:01.166186 15416 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 06 13:01:01 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
I20260806 13:01:01.169816 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.161662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45554)
0806 13:01:01.161705 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.161718 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:01.161762 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:01.162473 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:01.162474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:01.162629 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:01.162676 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.162679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.163573 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.163574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.164347 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.164350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.164855 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.164856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.164909 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.165342 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:01.165349 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:01.166893 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:01.168683 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.168684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.168685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.168931 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.169143 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.169144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.169144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.169214 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.169520 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:01.169521 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:01.169565 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 13:01:01.169631 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 13:01:01.169717 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 13:01:01.169842 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.162286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.24.83.60:39247)
0806 13:01:01.162345 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.162347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.162356 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:01:01.162374 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.162374 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.162389 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.162455 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.162781 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.163422 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.164477 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.164743 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.164805 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.166981 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.166982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.166982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.166990 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.168566 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.169011 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.169012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.169012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.169071 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.169283 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.169284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.169285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.169450 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.169524 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.169677 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.169753 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 13:01:01.171252 15330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8: Bootstrap starting.
I20260806 13:01:01.171847 15330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8: Neither blocks nor log segments found. Creating new log.
I20260806 13:01:01.172154 15330 log.cc:824] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8: Log is configured to *not* fsync() on all Append() calls
I20260806 13:01:01.172854 15330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8: No bootstrap required, opened a new log
I20260806 13:01:01.173979 15330 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } }
I20260806 13:01:01.174081 15330 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3ead5df328d4f66bba829b1c08dacd8, State: Initialized, Role: FOLLOWER
I20260806 13:01:01.174163 15330 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } }
I20260806 13:01:01.174463 15330 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:01:01.174490 15302 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 13:01:01.174559 15302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:01:01.175384 15302 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } }
I20260806 13:01:01.175622 15302 leader_election.cc:288] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [CANDIDATE]: Term 1 election: Requested vote from peers bb3410c2a2d24e4894a17cbde3335fdd (127.24.83.61:45423), 8dce7853c4144d0498bd74a942be969b (127.24.83.60:39247)
I20260806 13:01:01.175870 15459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3ead5df328d4f66bba829b1c08dacd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8dce7853c4144d0498bd74a942be969b"
I20260806 13:01:01.176047 15459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:01:01.177050 15459 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3ead5df328d4f66bba829b1c08dacd8 in term 1.
W20260806 13:01:01.176604 15520 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:01:01.177450 15520 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:01:01.177007 15507 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } } }
I20260806 13:01:01.177282 15275 leader_election.cc:302] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dce7853c4144d0498bd74a942be969b, e3ead5df328d4f66bba829b1c08dacd8; no voters:
I20260806 13:01:01.177791 15527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:01:01.177839 15507 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:01.178198 15527 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [term 1 LEADER]: Becoming Leader. State: Replica: e3ead5df328d4f66bba829b1c08dacd8, State: Running, Role: LEADER
I20260806 13:01:01.178283 15527 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } }
I20260806 13:01:01.178874 15507 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ead5df328d4f66bba829b1c08dacd8. Latest consensus state: current_term: 1 leader_uuid: "e3ead5df328d4f66bba829b1c08dacd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } } }
I20260806 13:01:01.178963 15507 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [sys.catalog]: This master's current role is: LEADER
I20260806 13:01:01.179167 15531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:01:01.179325 15531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:01:01.179495 15258 master_runner.cc:186] Error getting master registration for 127.24.83.61:45423: OK, instance_id { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" instance_seqno: 1786021260966142 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 13:01:01.181690 15258 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 13:01:01.188961 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.177499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42340 (local address 127.24.83.61:45423)
0806 13:01:01.177542 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.177544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.178604 (+ 1060us) server_negotiation.cc:407] Connection header received
0806 13:01:01.178660 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.178661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.178687 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.178744 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.180524 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.181210 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.182801 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.183100 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.183144 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.185508 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.185509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.185510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.185528 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.187152 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.187751 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.187753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.187754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.187833 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.188212 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.188214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.188214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.188363 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.188446 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.188760 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.188872 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:01:01.195839 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.178381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45564 (local address 127.24.83.60:39247)
0806 13:01:01.178419 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.178421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.188713 (+ 10292us) server_negotiation.cc:407] Connection header received
0806 13:01:01.188737 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.188738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.188748 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.188797 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.189386 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.190053 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.191190 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.191494 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.191534 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.192945 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.192946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.192947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.192953 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.194632 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.195015 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.195016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.195016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.195069 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.195268 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.195269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.195269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.195400 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.195477 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.195677 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.195751 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:01:01.197747 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.178075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.24.83.62:42255)
0806 13:01:01.178126 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.178128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.178737 (+ 609us) server_negotiation.cc:407] Connection header received
0806 13:01:01.181759 (+ 3022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.181759 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.181769 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.181824 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.182949 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.183656 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.184708 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.185010 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.185048 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.186981 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.186982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.186982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.186989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.188651 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.196925 (+ 8274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.196926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.196926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.196985 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.197176 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.197177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.197178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.197306 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.197372 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.197579 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.197656 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:01:01.199186 15459 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [term 1 FOLLOWER]: Refusing update from remote peer e3ead5df328d4f66bba829b1c08dacd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:01:01.199518 15507 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:01:01.200132 15407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } attempt: 3
I20260806 13:01:01.201186 15492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ead5df328d4f66bba829b1c08dacd8. Latest consensus state: current_term: 1 leader_uuid: "e3ead5df328d4f66bba829b1c08dacd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } } }
I20260806 13:01:01.201295 15492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 13:01:01 dist-test-slave-zkpd krb5kdc[15245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021260, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
I20260806 13:01:01.209087 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.200372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:39247 (local address 127.0.0.1:45578)
0806 13:01:01.200466 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:01.200476 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:01.200513 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:01.200684 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:01.200686 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:01.200861 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:01.200910 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.200913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.201789 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.201791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.202629 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:01.202632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.203090 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.203091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:01.203157 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.204359 (+ 1202us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:01.204364 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:01.205859 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:01.208010 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.208011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.208012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.208246 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.208463 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:01.208463 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:01.208464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:01.208534 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:01.208776 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:01.208788 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:01.208832 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 13:01:01.208895 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 13:01:01.208988 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 13:01:01.209090 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:01.200476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45578 (local address 127.24.83.60:39247)
0806 13:01:01.200511 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.200513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.200523 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:01:01.200580 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.200582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.200593 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.200640 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.200999 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.201698 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.202726 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:01.203017 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:01.203080 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:01.205948 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:01.205950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:01.205950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:01.205957 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:01.207916 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.208324 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.208325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.208325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.208406 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:01.208585 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:01.208585 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:01.208586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:01.208720 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:01.208776 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:01.208930 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 13:01:01.209003 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:01:01.210882 15407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd: Bootstrap starting.
I20260806 13:01:01.211621 15407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd: Neither blocks nor log segments found. Creating new log.
I20260806 13:01:01.211983 15407 log.cc:824] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd: Log is configured to *not* fsync() on all Append() calls
I20260806 13:01:01.212824 15407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd: No bootstrap required, opened a new log
I20260806 13:01:01.213114 15507 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3ead5df328d4f66bba829b1c08dacd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } } }
I20260806 13:01:01.213194 15507 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [sys.catalog]: This master's current role is: LEADER
I20260806 13:01:01.213694 15492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3ead5df328d4f66bba829b1c08dacd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } } }
I20260806 13:01:01.213794 15492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:01.213987 15532 mvcc.cc:205] Tried to move back new op lower bound from 7315543085869658112 to 7315543085787136000. Current Snapshot: MvccSnapshot[applied={T|T < 7315543085869658112}]
I20260806 13:01:01.215392 15407 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } }
I20260806 13:01:01.215559 15407 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb3410c2a2d24e4894a17cbde3335fdd, State: Initialized, Role: FOLLOWER
I20260806 13:01:01.215690 15407 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } }
I20260806 13:01:01.216039 15407 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:01:01.216090 15381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3ead5df328d4f66bba829b1c08dacd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bb3410c2a2d24e4894a17cbde3335fdd"
I20260806 13:01:01.216579 15381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:01:01.217410 15381 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3ead5df328d4f66bba829b1c08dacd8 in term 1.
I20260806 13:01:01.217800 15380 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [term 1 FOLLOWER]: Refusing update from remote peer e3ead5df328d4f66bba829b1c08dacd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:01:01.218416 15540 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } } }
I20260806 13:01:01.218555 15540 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:01.218560 15527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:01:01.220177 15540 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ead5df328d4f66bba829b1c08dacd8. Latest consensus state: current_term: 1 leader_uuid: "e3ead5df328d4f66bba829b1c08dacd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } } }
I20260806 13:01:01.220258 15540 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:01.220292 15540 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3ead5df328d4f66bba829b1c08dacd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } } }
I20260806 13:01:01.220332 15540 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:01.220371 15531 catalog_manager.cc:1347] Generated new cluster ID: 7f55a6b1c6a94c94838ec9e2fedd67cc
I20260806 13:01:01.220412 15531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260806 13:01:01.221485 15553 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:01:01.221539 15553 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:01:01.247234 15538 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3ead5df328d4f66bba829b1c08dacd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ead5df328d4f66bba829b1c08dacd8" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 42255 } } peers { permanent_uuid: "bb3410c2a2d24e4894a17cbde3335fdd" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 45423 } } peers { permanent_uuid: "8dce7853c4144d0498bd74a942be969b" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 39247 } } }
I20260806 13:01:01.247359 15538 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8 [sys.catalog]: This master's current role is: LEADER
I20260806 13:01:01.248333 15531 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:01:01.249150 15531 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:01:01.255023 15531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3ead5df328d4f66bba829b1c08dacd8: Generated new TSK 0
I20260806 13:01:01.255184 15531 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 13:01:01.304649 15412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.302798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42356 (local address 127.24.83.61:45423)
0806 13:01:01.302848 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.302850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.303918 (+ 1068us) server_negotiation.cc:407] Connection header received
0806 13:01:01.304040 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.304041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.304054 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.304087 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.304574 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42356: BlockingRecv error: recv got EOF from 127.0.0.1:42356 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 13:01:01.306371 15334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.303254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.24.83.62:42255)
0806 13:01:01.303298 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.303300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.305733 (+ 2433us) server_negotiation.cc:407] Connection header received
0806 13:01:01.305910 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.305911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.305924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.305955 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.306298 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33848: BlockingRecv error: recv got EOF from 127.0.0.1:33848 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 13:01:01.307461 15490 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.303562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.24.83.60:39247)
0806 13:01:01.303602 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.303604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.306669 (+ 3065us) server_negotiation.cc:407] Connection header received
0806 13:01:01.306852 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.306853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.306865 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.306910 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.307393 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45592: BlockingRecv error: recv got EOF from 127.0.0.1:45592 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 13:01:01.311374 15412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.310012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42372 (local address 127.24.83.61:45423)
0806 13:01:01.310087 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.310088 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.310413 (+ 325us) server_negotiation.cc:407] Connection header received
0806 13:01:01.310601 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.310602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.310625 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.310688 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.311300 (+ 612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42372: BlockingRecv error: recv got EOF from 127.0.0.1:42372 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 13:01:01.315151 15412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.314367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42380 (local address 127.24.83.61:45423)
0806 13:01:01.314405 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.314407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.314638 (+ 231us) server_negotiation.cc:407] Connection header received
0806 13:01:01.314733 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.314734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.314744 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.314786 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.315105 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42380: BlockingRecv error: recv got EOF from 127.0.0.1:42380 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 13:01:01.321034 15412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.320221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.24.83.61:45423)
0806 13:01:01.320260 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.320262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.320497 (+ 235us) server_negotiation.cc:407] Connection header received
0806 13:01:01.320599 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.320600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.320611 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.320641 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.320983 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42394: BlockingRecv error: recv got EOF from 127.0.0.1:42394 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 13:01:01.327129 15412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.326277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42400 (local address 127.24.83.61:45423)
0806 13:01:01.326314 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.326315 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.326585 (+ 270us) server_negotiation.cc:407] Connection header received
0806 13:01:01.326687 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.326688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.326699 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.326729 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.327081 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42400: BlockingRecv error: recv got EOF from 127.0.0.1:42400 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 13:01:01.337410 15412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.336486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42402 (local address 127.24.83.61:45423)
0806 13:01:01.336529 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.336531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.336826 (+ 295us) server_negotiation.cc:407] Connection header received
0806 13:01:01.336937 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.336938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.336950 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.336981 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.337350 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42402: BlockingRecv error: recv got EOF from 127.0.0.1:42402 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 13:01:01.355664 15412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.354760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42404 (local address 127.24.83.61:45423)
0806 13:01:01.354815 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.354817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.355098 (+ 281us) server_negotiation.cc:407] Connection header received
0806 13:01:01.355197 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.355198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.355210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.355239 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.355608 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42404: BlockingRecv error: recv got EOF from 127.0.0.1:42404 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 13:01:01.389832 15412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.389121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42406 (local address 127.24.83.61:45423)
0806 13:01:01.389191 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.389192 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.389235 (+ 43us) server_negotiation.cc:407] Connection header received
0806 13:01:01.389321 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.389322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.389335 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.389421 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.389770 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42406: BlockingRecv error: recv got EOF from 127.0.0.1:42406 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 13:01:01.455698 15412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.455019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42412 (local address 127.24.83.61:45423)
0806 13:01:01.455111 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.455113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.455206 (+ 93us) server_negotiation.cc:407] Connection header received
0806 13:01:01.455274 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.455275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.455286 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.455330 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.455632 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42412: BlockingRecv error: recv got EOF from 127.0.0.1:42412 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 13:01:01.585721 15412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.585024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42424 (local address 127.24.83.61:45423)
0806 13:01:01.585111 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.585113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.585184 (+ 71us) server_negotiation.cc:407] Connection header received
0806 13:01:01.585247 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.585248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.585258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.585322 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.585634 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42424: BlockingRecv error: recv got EOF from 127.0.0.1:42424 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 13:01:01.843837 15412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:01.843206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42432 (local address 127.24.83.61:45423)
0806 13:01:01.843289 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:01.843292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:01.843300 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:01:01.843329 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:01.843330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:01.843340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:01.843414 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:01.843755 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42432: BlockingRecv error: recv got EOF from 127.0.0.1:42432 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 13:01:02.164785 15504 catalog_manager.cc:1259] Loaded cluster ID: 7f55a6b1c6a94c94838ec9e2fedd67cc
I20260806 13:01:02.164865 15504 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b: loading cluster ID for follower catalog manager: success
I20260806 13:01:02.165627 15504 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b: acquiring CA information for follower catalog manager: success
I20260806 13:01:02.165869 15504 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8dce7853c4144d0498bd74a942be969b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 13:01:02.221927 15553 catalog_manager.cc:1259] Loaded cluster ID: 7f55a6b1c6a94c94838ec9e2fedd67cc
I20260806 13:01:02.222007 15553 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd: loading cluster ID for follower catalog manager: success
I20260806 13:01:02.222788 15553 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd: acquiring CA information for follower catalog manager: success
I20260806 13:01:02.223032 15553 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb3410c2a2d24e4894a17cbde3335fdd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 13:01:02.359539 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:02.357147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42446 (local address 127.24.83.61:45423)
0806 13:01:02.357256 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:02.357257 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:02.357267 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:01:02.357328 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:02.357329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:02.357392 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:02.357477 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:01:02.357619 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:02.358111 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:02.358858 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:02.359044 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:02.359094 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:02.359247 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:01:02.359355 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:01:02.359420 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:02.359452 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 13:01:02.359470 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
I20260806 13:01:02.363996 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:02.361514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45606 (local address 127.24.83.60:39247)
0806 13:01:02.361637 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:02.361639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:02.361653 (+ 14us) server_negotiation.cc:407] Connection header received
0806 13:01:02.361692 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:02.361692 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:02.361729 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:02.361797 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:01:02.361910 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:02.362498 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:02.363246 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:02.363431 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:02.363511 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:02.363634 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:01:02.363771 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:01:02.363843 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:02.363912 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 13:01:02.363930 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":80,"threads_started":1}
I20260806 13:01:02.368925 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:02.365687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42458 (local address 127.24.83.61:45423)
0806 13:01:02.365729 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:02.365731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:02.366018 (+ 287us) server_negotiation.cc:407] Connection header received
0806 13:01:02.366084 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:02.366085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:02.366096 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:02.366147 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:01:02.366262 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:02.366995 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:02.368102 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:02.368431 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:02.368498 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:02.368633 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:01:02.368671 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:01:02.368733 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:02.368797 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 13:01:02.368821 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:01:02.369534 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:02.365889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45612 (local address 127.24.83.60:39247)
0806 13:01:02.365949 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:02.365951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:02.366086 (+ 135us) server_negotiation.cc:407] Connection header received
0806 13:01:02.366110 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:02.366111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:02.366120 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:02.366168 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:01:02.367177 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:02.367674 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:02.368769 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:02.369057 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:02.369114 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:02.369277 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:01:02.369312 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:01:02.369361 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:02.369433 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 13:01:02.369454 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 13:01:02.369675 15639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:02.365752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33854 (local address 127.24.83.62:42255)
0806 13:01:02.365865 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:02.365870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:02.365882 (+ 12us) server_negotiation.cc:407] Connection header received
0806 13:01:02.365945 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:02.365946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:02.365995 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:02.366083 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:01:02.366341 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:02.367083 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:02.368541 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:02.368857 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:02.369070 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:02.369150 (+ 80us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:01:02.369277 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:01:02.369528 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:02.369551 (+ 23us) server_negotiation.cc:299] Negotiation successful
0806 13:01:02.369582 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
I20260806 13:01:02.371307 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 15258
I20260806 13:01:02.378264 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 15338
I20260806 13:01:02.383694 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 15416
2026-08-06T13:01:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2836 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:01:02 dist-test-slave-zkpd krb5kdc[15643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:01:02 dist-test-slave-zkpd krb5kdc[15643](info): set up 2 sockets
Aug 06 13:01:02 dist-test-slave-zkpd krb5kdc[15643](info): commencing operation
krb5kdc: starting...
W20260806 13:01:03.424228 24908 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/test-user.keytab.
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:01:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:01:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.83.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
Entry for principal kudu/127.24.83.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab.
I20260806 13:01:03.540616 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:38263
--webserver_interface=127.24.83.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:38263,127.24.83.61:32955,127.24.83.60:40729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:01:03.621405 15657 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:01:03.621642 15657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:01:03.621672 15657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:01:03.621716 15657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:01:03.622821 15657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:01:03.622889 15657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:01:03.622913 15657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:01:03.622936 15657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:01:03.622965 15657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:01:03.624452 15657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:38263,127.24.83.61:32955,127.24.83.60:40729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.62.keytab
--principal=kudu/127.24.83.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.62:38263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.83.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:01:03.624729 15657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:01:03.624954 15657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:01:03.629706 15666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:01:03.629741 15664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:01:03.629748 15657 server_base.cc:1034] running on GCE node
W20260806 13:01:03.629703 15663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:01:03.630136 15657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:01:03.630424 15657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:01:03.631573 15657 hybrid_clock.cc:648] HybridClock initialized: now 1786021263631549 us; error 35 us; skew 500 ppm
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:01:03.633997 15657 init.cc:377] Logged in from keytab as kudu/127.24.83.62@KRBTEST.COM (short username kudu)
I20260806 13:01:03.634470 15657 webserver.cc:460] Webserver started at http://127.24.83.62:39589/ using document root <none> and password file <none>
I20260806 13:01:03.634716 15657 fs_manager.cc:359] Metadata directory not provided
I20260806 13:01:03.634791 15657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:01:03.634936 15657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:01:03.636014 15657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/data/instance:
uuid: "bca5cd66662c439c847bd1bc84f70d9b"
format_stamp: "Formatted at 2026-08-06 13:01:03 on dist-test-slave-zkpd"
server_key: "4874246fa8782cdb6edac60fcf15a625"
server_key_iv: "34bc9a2e0d0f4d7a101a3a56c7bdb79b"
server_key_version: "encryptionkey@0"
I20260806 13:01:03.636364 15657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/wal/instance:
uuid: "bca5cd66662c439c847bd1bc84f70d9b"
format_stamp: "Formatted at 2026-08-06 13:01:03 on dist-test-slave-zkpd"
server_key: "4874246fa8782cdb6edac60fcf15a625"
server_key_iv: "34bc9a2e0d0f4d7a101a3a56c7bdb79b"
server_key_version: "encryptionkey@0"
I20260806 13:01:03.637645 15657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:01:03.638330 15673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:01:03.638496 15657 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:01:03.638599 15657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/wal
uuid: "bca5cd66662c439c847bd1bc84f70d9b"
format_stamp: "Formatted at 2026-08-06 13:01:03 on dist-test-slave-zkpd"
server_key: "4874246fa8782cdb6edac60fcf15a625"
server_key_iv: "34bc9a2e0d0f4d7a101a3a56c7bdb79b"
server_key_version: "encryptionkey@0"
I20260806 13:01:03.638672 15657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:01:03.655454 15657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:01:03.657465 15657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:01:03.657626 15657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:01:03.661801 15657 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.62:38263
I20260806 13:01:03.661837 15728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.62:38263 every 8 connection(s)
I20260806 13:01:03.662178 15657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/data/info.pb
I20260806 13:01:03.663511 15729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:01:03.665153 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 15657
I20260806 13:01:03.665236 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-0/wal/instance
I20260806 13:01:03.665153 15729 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:01:03.665437 24908 external_mini_cluster.cc:1428] Setting key 625e0e45825206f144f0ec25e53f8c0f
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.83.61@KRBTEST.COM; defaulting to no policy
I20260806 13:01:03.674963 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.665858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:38263 (local address 127.0.0.1:51094)
0806 13:01:03.666066 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.666110 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:03.666834 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:03.667170 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:03.667172 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:03.667409 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:03.667560 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.667564 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.668732 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.668733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.669386 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.669388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.669919 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.669920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.669985 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.670504 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:03.670512 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:03.671802 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:03.673580 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.673589 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.673591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.673879 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.674096 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.674097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.674097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.674175 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.674512 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:03.674518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:03.674574 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 13:01:03.674648 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:01:03.674768 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":119,"threads_started":1}
I20260806 13:01:03.674968 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.665932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51094 (local address 127.24.83.62:38263)
0806 13:01:03.666094 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:03.666102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:03.666211 (+ 109us) server_negotiation.cc:407] Connection header received
0806 13:01:03.667008 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:03.667011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:03.667035 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:03.667142 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:03.667656 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.668653 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.669487 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.669800 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.669925 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.671910 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:03.671918 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:03.671918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:03.671925 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:03.673467 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.673956 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.673959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.673959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.674034 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.674231 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.674232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.674232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.674445 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:03.674526 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:03.674686 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 13:01:03.674822 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260806 13:01:03.675689 15729 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.61@KRBTEST.COM" created.
I20260806 13:01:03.676087 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.675952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:32955 (local address 127.0.0.1:40422)
0806 13:01:03.676003 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.676054 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:32955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 13:01:03.676219 15676 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.83.61:32955: connect: Connection refused (error 111)
I20260806 13:01:03.676651 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.676556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:32955 (local address 127.0.0.1:40426)
0806 13:01:03.676604 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.676628 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:32955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 13:01:03.676767 15729 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.61:32955: Network error: Client connection negotiation failed: client connection to 127.24.83.61:32955: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab.
Entry for principal kudu/127.24.83.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab.
I20260806 13:01:03.690364 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.61:32955
--webserver_interface=127.24.83.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab
--principal=kudu/127.24.83.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:38263,127.24.83.61:32955,127.24.83.60:40729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:01:03.725999 15729 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } attempt: 1
I20260806 13:01:03.726542 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.726404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:32955 (local address 127.0.0.1:40440)
0806 13:01:03.726472 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.726513 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:32955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 13:01:03.726933 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.726857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:32955 (local address 127.0.0.1:40446)
0806 13:01:03.726895 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.726917 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:32955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 13:01:03.727035 15729 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.61:32955: Network error: Client connection negotiation failed: client connection to 127.24.83.61:32955: connect: Connection refused (error 111)
W20260806 13:01:03.771299 15737 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:01:03.771510 15737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:01:03.771528 15737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:01:03.771548 15737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:01:03.772639 15737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:01:03.772673 15737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:01:03.772687 15737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:01:03.772701 15737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:01:03.772712 15737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:01:03.774129 15737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:38263,127.24.83.61:32955,127.24.83.60:40729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.61.keytab
--principal=kudu/127.24.83.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.61:32955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.83.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:01:03.774374 15737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:01:03.774570 15737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:01:03.779222 15742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:01:03.779228 15745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:01:03.779222 15743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:01:03.779464 15737 server_base.cc:1034] running on GCE node
I20260806 13:01:03.779680 15737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:01:03.779991 15737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:01:03.781132 15737 hybrid_clock.cc:648] HybridClock initialized: now 1786021263781123 us; error 25 us; skew 500 ppm
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:01:03.783618 15737 init.cc:377] Logged in from keytab as kudu/127.24.83.61@KRBTEST.COM (short username kudu)
I20260806 13:01:03.784108 15737 webserver.cc:460] Webserver started at http://127.24.83.61:42931/ using document root <none> and password file <none>
I20260806 13:01:03.784343 15737 fs_manager.cc:359] Metadata directory not provided
I20260806 13:01:03.784436 15737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:01:03.784584 15737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:01:03.785631 15737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/data/instance:
uuid: "cbf2a91ea2f641f89fcaf6136f89820c"
format_stamp: "Formatted at 2026-08-06 13:01:03 on dist-test-slave-zkpd"
server_key: "db6d8cd757307fe56cd5f0d55ef96eab"
server_key_iv: "39fa7158ffd35779861c07a019ee294d"
server_key_version: "encryptionkey@0"
I20260806 13:01:03.785996 15737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/wal/instance:
uuid: "cbf2a91ea2f641f89fcaf6136f89820c"
format_stamp: "Formatted at 2026-08-06 13:01:03 on dist-test-slave-zkpd"
server_key: "db6d8cd757307fe56cd5f0d55ef96eab"
server_key_iv: "39fa7158ffd35779861c07a019ee294d"
server_key_version: "encryptionkey@0"
I20260806 13:01:03.787328 15737 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:01:03.788085 15752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:01:03.788261 15737 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:01:03.788362 15737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/wal
uuid: "cbf2a91ea2f641f89fcaf6136f89820c"
format_stamp: "Formatted at 2026-08-06 13:01:03 on dist-test-slave-zkpd"
server_key: "db6d8cd757307fe56cd5f0d55ef96eab"
server_key_iv: "39fa7158ffd35779861c07a019ee294d"
server_key_version: "encryptionkey@0"
I20260806 13:01:03.788450 15737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:01:03.795188 15729 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } attempt: 2
I20260806 13:01:03.795601 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.795481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:32955 (local address 127.0.0.1:40460)
0806 13:01:03.795538 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.795574 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:32955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 13:01:03.795969 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.795861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:32955 (local address 127.0.0.1:40464)
0806 13:01:03.795919 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.795951 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.61:32955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 13:01:03.796051 15729 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.61:32955: Network error: Client connection negotiation failed: client connection to 127.24.83.61:32955: connect: Connection refused (error 111)
I20260806 13:01:03.809573 15737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:01:03.811626 15737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:01:03.811790 15737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:01:03.816169 15737 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.61:32955
I20260806 13:01:03.816221 15805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.61:32955 every 8 connection(s)
I20260806 13:01:03.816592 15737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/data/info.pb
I20260806 13:01:03.817555 15806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:01:03.819140 15806 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:01:03.825110 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 15737
I20260806 13:01:03.825224 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-1/wal/instance
I20260806 13:01:03.825409 24908 external_mini_cluster.cc:1428] Setting key f147a6fd7d1a55cf46ffdaff74d34481
I20260806 13:01:03.827849 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.819833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51108 (local address 127.24.83.62:38263)
0806 13:01:03.819937 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:03.819939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:03.820115 (+ 176us) server_negotiation.cc:407] Connection header received
0806 13:01:03.820564 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:03.820565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:03.820578 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:03.820629 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:03.821177 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.821640 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.822561 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.822748 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.822811 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.824765 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:03.824767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:03.824767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:03.824773 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:03.826339 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.826846 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.826847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.826847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.826916 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.827184 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.827186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.827186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.827346 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:03.827415 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:03.827648 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 13:01:03.827739 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 13:01:03.827877 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.819784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:38263 (local address 127.0.0.1:51108)
0806 13:01:03.819997 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.820034 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:03.820484 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:03.820680 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:03.820683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:03.820921 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:03.821109 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.821113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.821717 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.821718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.822490 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.822492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.822842 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.822843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.822951 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.823362 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:03.823370 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:03.824675 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:03.826423 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.826431 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.826433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.826763 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.826998 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.826999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.826999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.827122 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.827412 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:03.827417 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:03.827528 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 13:01:03.827607 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 13:01:03.827726 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":125,"threads_started":1}
I20260806 13:01:03.828523 15806 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.83.60@KRBTEST.COM; defaulting to no policy
I20260806 13:01:03.835618 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.828782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:32955 (local address 127.0.0.1:40472)
0806 13:01:03.828833 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.828843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:03.828881 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:03.829157 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:03.829158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:03.829291 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:03.829340 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.829342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.830187 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.830188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.830677 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.830679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.831149 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.831150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.831215 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.831609 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:03.831614 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:03.833032 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:03.834702 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.834703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.834703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.834862 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.835044 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.835044 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.835045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.835093 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.835400 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:03.835400 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:03.835434 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 13:01:03.835483 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 13:01:03.835556 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 13:01:03.835815 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.828849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.24.83.61:32955)
0806 13:01:03.829013 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:03.829017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:03.829034 (+ 17us) server_negotiation.cc:407] Connection header received
0806 13:01:03.829050 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:03.829051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:03.829061 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:03.829144 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:03.829420 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.830092 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.830766 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.831065 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.831140 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.833113 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:03.833121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:03.833121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:03.833127 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:03.834633 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.834933 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.834935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.834936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.834987 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.835131 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.835132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.835132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.835344 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:03.835412 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:03.835513 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 13:01:03.835614 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.83.60@KRBTEST.COM" created.
I20260806 13:01:03.836433 15806 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:01:03.836787 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.836645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:47896)
0806 13:01:03.836695 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.836746 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 13:01:03.836968 15756 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
I20260806 13:01:03.837390 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.837285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:47908)
0806 13:01:03.837331 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.837361 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 13:01:03.837536 15806 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:40729: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.83.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab.
Entry for principal kudu/127.24.83.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab.
I20260806 13:01:03.848932 24908 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzNDPsb/build/release/bin/kudu
/tmp/dist-test-taskzNDPsb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.60:40729
--webserver_interface=127.24.83.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab
--principal=kudu/127.24.83.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.83.20:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.83.62:38263,127.24.83.61:32955,127.24.83.60:40729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:01:03.886785 15806 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } attempt: 1
I20260806 13:01:03.887193 15729 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } attempt: 3
I20260806 13:01:03.887336 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.887135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:47914)
0806 13:01:03.887242 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.887292 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 13:01:03.887790 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.887669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:47930)
0806 13:01:03.887706 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.887737 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 13:01:03.888200 15806 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:40729: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
I20260806 13:01:03.894681 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.887480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:32955 (local address 127.0.0.1:40474)
0806 13:01:03.887543 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.887553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:03.887602 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:03.887804 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:03.887806 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:03.888008 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:03.888082 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.888084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.889016 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.889017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.889666 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.889669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.890162 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.890162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.890218 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.890700 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:03.890705 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:03.892021 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:03.893720 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.893721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.893722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.893918 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.894135 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.894136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.894136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.894187 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.894450 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:03.894451 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:03.894489 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 13:01:03.894540 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:01:03.894614 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 13:01:03.894737 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.887539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.24.83.61:32955)
0806 13:01:03.887579 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:03.887580 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:03.887590 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:01:03.887684 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:03.887686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:03.887699 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:03.887778 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:03.888163 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.888937 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.889747 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.890083 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.890150 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.892097 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:03.892098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:03.892099 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:03.892106 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:03.893620 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.893993 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.893995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.893995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.894065 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.894245 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.894246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.894247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.894390 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:03.894456 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:03.894581 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 13:01:03.894656 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:01:03.895139 15729 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:01:03.895475 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.895363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:47936)
0806 13:01:03.895420 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.895451 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 13:01:03.895772 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.895690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:47944)
0806 13:01:03.895730 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.895754 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 13:01:03.895851 15729 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:40729: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
I20260806 13:01:03.927006 15729 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } attempt: 1
I20260806 13:01:03.927534 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.927372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:47956)
0806 13:01:03.927453 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.927496 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 13:01:03.927966 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.927835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:47972)
0806 13:01:03.927886 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.927944 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 13:01:03.928084 15729 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:40729: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
W20260806 13:01:03.932610 15813 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:01:03.932803 15813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:01:03.932845 15813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:01:03.932890 15813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:01:03.934017 15813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:01:03.934057 15813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:01:03.934072 15813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:01:03.934104 15813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:01:03.934130 15813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:01:03.935546 15813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.83.20:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.83.62:38263,127.24.83.61:32955,127.24.83.60:40729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/krb5kdc/kudu_127.24.83.60.keytab
--principal=kudu/127.24.83.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.83.60:40729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.83.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:01:03.935783 15813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:01:03.936041 15813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:01:03.940685 15821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:01:03.940676 15819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:01:03.940706 15813 server_base.cc:1034] running on GCE node
W20260806 13:01:03.940675 15818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:01:03.941066 15813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:01:03.941409 15813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:01:03.942566 15813 hybrid_clock.cc:648] HybridClock initialized: now 1786021263942554 us; error 33 us; skew 500 ppm
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:01:03.945181 15813 init.cc:377] Logged in from keytab as kudu/127.24.83.60@KRBTEST.COM (short username kudu)
I20260806 13:01:03.945607 15813 webserver.cc:460] Webserver started at http://127.24.83.60:36153/ using document root <none> and password file <none>
I20260806 13:01:03.945828 15813 fs_manager.cc:359] Metadata directory not provided
I20260806 13:01:03.945873 15813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:01:03.946029 15813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:01:03.947082 15813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/data/instance:
uuid: "524c640dec07441cb9b7f93e1cc67a62"
format_stamp: "Formatted at 2026-08-06 13:01:03 on dist-test-slave-zkpd"
server_key: "bd6412d4a38eabdbb96d9cfa326b45c1"
server_key_iv: "519894d31d837bd819cfbb884ef456a1"
server_key_version: "encryptionkey@0"
I20260806 13:01:03.947450 15813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/wal/instance:
uuid: "524c640dec07441cb9b7f93e1cc67a62"
format_stamp: "Formatted at 2026-08-06 13:01:03 on dist-test-slave-zkpd"
server_key: "bd6412d4a38eabdbb96d9cfa326b45c1"
server_key_iv: "519894d31d837bd819cfbb884ef456a1"
server_key_version: "encryptionkey@0"
I20260806 13:01:03.948841 15813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:01:03.949536 15828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:01:03.949728 15813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:01:03.949813 15813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/data,/tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/wal
uuid: "524c640dec07441cb9b7f93e1cc67a62"
format_stamp: "Formatted at 2026-08-06 13:01:03 on dist-test-slave-zkpd"
server_key: "bd6412d4a38eabdbb96d9cfa326b45c1"
server_key_iv: "519894d31d837bd819cfbb884ef456a1"
server_key_version: "encryptionkey@0"
I20260806 13:01:03.949865 15813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:01:03.956375 15806 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } attempt: 2
I20260806 13:01:03.956754 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.956628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:47988)
0806 13:01:03.956689 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.956726 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 13:01:03.957080 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.956988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:48004)
0806 13:01:03.957031 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.957055 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 13:01:03.957175 15806 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.83.60:40729: Network error: Client connection negotiation failed: client connection to 127.24.83.60:40729: connect: Connection refused (error 111)
I20260806 13:01:03.959182 15813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:01:03.960985 15813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:01:03.961141 15813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:01:03.965075 15813 rpc_server.cc:307] RPC server started. Bound to: 127.24.83.60:40729
I20260806 13:01:03.965106 15883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.83.60:40729 every 8 connection(s)
I20260806 13:01:03.965445 15813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/data/info.pb
I20260806 13:01:03.966426 15884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:01:03.967991 15884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:01:03.973417 24908 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzNDPsb/build/release/bin/kudu as pid 15813
I20260806 13:01:03.973526 24908 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzNDPsb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786020991335182-24908-0/minicluster-data/master-2/wal/instance
I20260806 13:01:03.973750 24908 external_mini_cluster.cc:1428] Setting key 974e38fe89a481f19347b6d018416feb
I20260806 13:01:03.976814 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.968662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51116 (local address 127.24.83.62:38263)
0806 13:01:03.968711 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:03.968713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:03.968899 (+ 186us) server_negotiation.cc:407] Connection header received
0806 13:01:03.969338 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:03.969339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:03.969347 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:03.969403 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:03.969940 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.970399 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.971299 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.971471 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.971521 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.973673 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:03.973674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:03.973675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:03.973692 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:03.975323 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.975787 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.975788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.975788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.975848 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.976161 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.976162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.976162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.976271 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:03.976322 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:03.976636 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 13:01:03.976719 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:01:03.976920 15886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.968611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.62:38263 (local address 127.0.0.1:51116)
0806 13:01:03.968772 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.968814 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:03.969261 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:03.969456 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:03.969459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:03.969686 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:03.969870 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.969873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.970476 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.970476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.971224 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.971226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.971552 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.971553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.971654 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.972125 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:03.972133 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:03.973570 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:03.975406 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.975412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.975413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.975705 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.975966 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.975967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.975967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.976094 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.976334 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:03.976338 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:03.976483 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 13:01:03.976591 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 13:01:03.976734 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260806 13:01:03.977497 15884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
I20260806 13:01:03.983671 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.974155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48008 (local address 127.24.83.60:40729)
0806 13:01:03.974321 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:03.974326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:03.974415 (+ 89us) server_negotiation.cc:407] Connection header received
0806 13:01:03.974661 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:03.974666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:03.974678 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:03.974740 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:03.975505 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.976543 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.977592 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.978006 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.978061 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.980656 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:03.980665 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:03.980665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:03.980672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:03.982368 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.982780 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.982783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.982783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.982864 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.983066 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.983067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.983068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.983212 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:03.983357 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:03.983479 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 13:01:03.983553 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
I20260806 13:01:03.984859 15886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.977776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.61:32955 (local address 127.0.0.1:40478)
0806 13:01:03.977824 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.977833 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:03.977869 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:03.977997 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:03.977997 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:03.978140 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:03.978200 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.978201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.978922 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.978924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.979706 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.979708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.980259 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.980260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.980338 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.980706 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:03.980710 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:03.981862 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:03.983532 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.983533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.983533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.983748 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.984039 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.984040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.984040 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.984100 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.984558 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:03.984558 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:03.984603 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 13:01:03.984676 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 13:01:03.984780 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 13:01:03.984925 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.977851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.24.83.61:32955)
0806 13:01:03.977902 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:03.977905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:03.977913 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:01:03.977937 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:03.977938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:03.977947 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:03.977996 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:03.978306 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.978839 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.979790 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.980141 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.980205 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.981958 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:03.981960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:03.981960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:03.981967 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:03.983447 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.983869 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.983870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.983871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.983956 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.984168 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.984169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.984170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.984301 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:03.984402 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:03.984720 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 13:01:03.984799 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:01:03.985234 15884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.60@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.62@KRBTEST.COM
I20260806 13:01:03.992409 15886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.985490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:48014)
0806 13:01:03.985555 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:03.985564 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:03.985599 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:03.985702 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:03.985703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:03.985839 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:03.985870 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.985871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.986513 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.986514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.986991 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:03.986993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.987360 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.987361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:03.987397 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.987803 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:03.987807 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:03.989618 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:03.991333 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.991334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.991335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.991540 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.991728 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:03.991729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:03.991729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:03.991792 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:03.992097 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:03.992098 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:03.992142 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 13:01:03.992194 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 13:01:03.992289 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 13:01:03.992635 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.984577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51128 (local address 127.24.83.62:38263)
0806 13:01:03.984620 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:03.984623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:03.984631 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:01:03.984674 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:03.984675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:03.984685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:03.984735 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:03.985678 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.986244 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.986957 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.987202 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.987250 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.989895 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:03.989897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:03.989897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:03.989904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:03.991414 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.991786 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.991787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.991787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.991855 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.992073 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.992074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.992074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.992223 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:03.992388 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:03.992462 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 13:01:03.992546 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:01:03.992702 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.985539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48014 (local address 127.24.83.60:40729)
0806 13:01:03.985577 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:03.985578 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:03.985590 (+ 12us) server_negotiation.cc:407] Connection header received
0806 13:01:03.985638 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:03.985639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:03.985649 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:03.985700 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:03.985939 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.986435 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.987068 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.987280 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.987340 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.989700 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:03.989701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:03.989702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:03.989707 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:03.991195 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.991616 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.991617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.991617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.991664 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:03.991858 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:03.991859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:03.991859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:03.992008 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:03.992094 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:03.992237 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 13:01:03.992348 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":22}
I20260806 13:01:03.994257 15884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62: Bootstrap starting.
I20260806 13:01:03.995172 15884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62: Neither blocks nor log segments found. Creating new log.
I20260806 13:01:03.995487 15884 log.cc:824] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62: Log is configured to *not* fsync() on all Append() calls
I20260806 13:01:03.996428 15884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62: No bootstrap required, opened a new log
Aug 06 13:01:03 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.83.61@KRBTEST.COM
I20260806 13:01:03.998297 15884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } }
I20260806 13:01:03.998469 15884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524c640dec07441cb9b7f93e1cc67a62, State: Initialized, Role: FOLLOWER
I20260806 13:01:03.998571 15884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } }
I20260806 13:01:03.998944 15884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:01:03.998898 15890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } } }
I20260806 13:01:03.999024 15890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 13:01:04.000916 15901 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:01:04.000974 15901 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:01:04.001276 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:03.993347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40482 (local address 127.24.83.61:32955)
0806 13:01:03.993393 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:03.993395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:03.993405 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:01:03.993447 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:03.993448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:03.993458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:03.993514 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:03.993950 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.994623 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.995259 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:03.995567 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:03.995635 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:03.998014 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:03.998015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:03.998016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:03.998023 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:03.999763 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.000151 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.000153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.000153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.000212 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.000535 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.000537 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.000537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.000682 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:04.000866 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:04.001089 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 13:01:04.001169 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:01:04.002795 24908 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:01:04.003271 15729 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } attempt: 2
I20260806 13:01:04.001339 15813 master_runner.cc:186] Error getting master registration for 127.24.83.62:38263: OK, instance_id { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" instance_seqno: 1786021263660620 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 13:01:04.004350 15813 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 13:01:04.004143 15703 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bca5cd66662c439c847bd1bc84f70d9b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51128
Aug 06 13:01:04 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.62@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
I20260806 13:01:04.010977 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:04.003701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48024 (local address 127.24.83.60:40729)
0806 13:01:04.003765 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.003767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.003776 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:01:04.003796 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.003797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.003806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.003870 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.004256 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.004772 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.005595 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.005891 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.005955 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:04.008083 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:04.008084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:04.008085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:04.008091 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:04.009717 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.010116 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.010117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.010117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.010172 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.010375 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.010375 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.010376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.010534 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:04.010600 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:04.010805 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 13:01:04.010883 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 13:01:04.011019 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:04.003501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:48024)
0806 13:01:04.003543 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:04.003556 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:04.003603 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:04.003953 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:04.003954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:04.004117 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:04.004169 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:04.004171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.004860 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.004861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:04.005506 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:04.005509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.005971 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.005972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:04.006028 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:04.006460 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:04.006464 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:04.007982 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:04.009811 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:04.009812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:04.009813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:04.010040 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:04.010241 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:04.010242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:04.010242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:04.010308 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:04.010611 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:04.010612 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:04.010675 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 13:01:04.010767 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 13:01:04.010891 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 13:01:04.012490 15729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b: Bootstrap starting.
I20260806 13:01:04.013105 15729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b: Neither blocks nor log segments found. Creating new log.
I20260806 13:01:04.013399 15729 log.cc:824] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b: Log is configured to *not* fsync() on all Append() calls
I20260806 13:01:04.014145 15729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b: No bootstrap required, opened a new log
I20260806 13:01:04.015116 15729 raft_consensus.cc:348] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } }
I20260806 13:01:04.015246 15729 raft_consensus.cc:729] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca5cd66662c439c847bd1bc84f70d9b, State: Initialized, Role: FOLLOWER
I20260806 13:01:04.015374 15729 consensus_queue.cc:260] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } }
I20260806 13:01:04.015739 15729 sys_catalog.cc:564] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:01:04.015769 15703 raft_consensus.cc:482] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 13:01:04.015813 15703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:01:04.016662 15703 raft_consensus.cc:504] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } }
I20260806 13:01:04.016944 15703 leader_election.cc:288] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [CANDIDATE]: Term 1 election: Requested vote from peers cbf2a91ea2f641f89fcaf6136f89820c (127.24.83.61:32955), 524c640dec07441cb9b7f93e1cc67a62 (127.24.83.60:40729)
I20260806 13:01:04.015697 15904 sys_catalog.cc:455] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } } }
I20260806 13:01:04.017325 15904 sys_catalog.cc:458] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:04.017618 15858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bca5cd66662c439c847bd1bc84f70d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "524c640dec07441cb9b7f93e1cc67a62"
I20260806 13:01:04.017743 15858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [term 0 FOLLOWER]: Advancing to term 1
W20260806 13:01:04.018008 15916 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:01:04.018050 15916 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:01:04.018697 15858 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca5cd66662c439c847bd1bc84f70d9b in term 1.
I20260806 13:01:04.019232 15677 leader_election.cc:302] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524c640dec07441cb9b7f93e1cc67a62, bca5cd66662c439c847bd1bc84f70d9b; no voters:
I20260806 13:01:04.019673 15904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:01:04.019923 15904 raft_consensus.cc:686] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [term 1 LEADER]: Becoming Leader. State: Replica: bca5cd66662c439c847bd1bc84f70d9b, State: Running, Role: LEADER
I20260806 13:01:04.020018 15904 consensus_queue.cc:237] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } }
I20260806 13:01:04.020489 15922 sys_catalog.cc:455] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bca5cd66662c439c847bd1bc84f70d9b. Latest consensus state: current_term: 1 leader_uuid: "bca5cd66662c439c847bd1bc84f70d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } } }
I20260806 13:01:04.020566 15922 sys_catalog.cc:458] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [sys.catalog]: This master's current role is: LEADER
I20260806 13:01:04.020749 15924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:01:04.020910 15924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:01:04.020668 15657 master_runner.cc:186] Error getting master registration for 127.24.83.61:32955: OK, instance_id { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" instance_seqno: 1786021263814805 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 13:01:04.023089 15657 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 13:01:04.024274 15858 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [term 1 FOLLOWER]: Refusing update from remote peer bca5cd66662c439c847bd1bc84f70d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:01:04.024588 15904 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:01:04.025799 15890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bca5cd66662c439c847bd1bc84f70d9b. Latest consensus state: current_term: 1 leader_uuid: "bca5cd66662c439c847bd1bc84f70d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } } }
I20260806 13:01:04.025885 15890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:04.026897 15904 sys_catalog.cc:455] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bca5cd66662c439c847bd1bc84f70d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } } }
I20260806 13:01:04.026996 15904 sys_catalog.cc:458] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [sys.catalog]: This master's current role is: LEADER
I20260806 13:01:04.027314 15890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bca5cd66662c439c847bd1bc84f70d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } } }
I20260806 13:01:04.027402 15890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:04.027585 15925 mvcc.cc:205] Tried to move back new op lower bound from 7315543097440894976 to 7315543097426739200. Current Snapshot: MvccSnapshot[applied={T|T < 7315543097440894976}]
I20260806 13:01:04.028110 15928 mvcc.cc:205] Tried to move back new op lower bound from 7315543097440894976 to 7315543097426739200. Current Snapshot: MvccSnapshot[applied={T|T < 7315543097440894976}]
I20260806 13:01:04.028573 15924 catalog_manager.cc:1347] Generated new cluster ID: 1317dfe5df384bf7970e57b52a8471bb
I20260806 13:01:04.028614 15924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:01:04.035192 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:04.023415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51140 (local address 127.24.83.62:38263)
0806 13:01:04.023457 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.023460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.023980 (+ 520us) server_negotiation.cc:407] Connection header received
0806 13:01:04.024132 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.024133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.024155 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.024186 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.025632 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.026376 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.028614 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.028923 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.028965 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:04.031533 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:04.031534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:04.031535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:04.031542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:04.033229 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.033955 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.033957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.033957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.034016 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.034380 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.034381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.034381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.034525 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:04.034617 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:04.035013 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 13:01:04.035097 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:01:04.043833 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:04.023626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40486 (local address 127.24.83.61:32955)
0806 13:01:04.023675 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.023687 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.028579 (+ 4892us) server_negotiation.cc:407] Connection header received
0806 13:01:04.028848 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.028850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.028871 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.028902 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.036466 (+ 7564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.037209 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.038195 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.038505 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.038548 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:04.040079 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:04.040081 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:04.040081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:04.040089 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:04.042130 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.042714 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.042716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.042716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.042783 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.043093 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.043094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.043095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.043235 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:04.043316 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:04.043633 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 13:01:04.043721 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:01:04.048241 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:04.027506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48038 (local address 127.24.83.60:40729)
0806 13:01:04.027543 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.027545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.036477 (+ 8932us) server_negotiation.cc:407] Connection header received
0806 13:01:04.036662 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.036663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.036674 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.036704 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.037850 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.038517 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.040141 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.040430 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.040471 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:04.044752 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:04.044753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:04.044753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:04.044769 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:04.046397 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.047044 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.047045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.047046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.047101 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.047433 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.047434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.047434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.047561 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:04.047654 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:04.048030 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 13:01:04.048134 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:01:04.048301 15806 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } attempt: 3
Aug 06 13:01:04 dist-test-slave-zkpd krb5kdc[15643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.83.61@KRBTEST.COM for kudu/127.24.83.60@KRBTEST.COM
I20260806 13:01:04.057193 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:04.048569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.83.60:40729 (local address 127.0.0.1:48040)
0806 13:01:04.048649 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 13:01:04.048659 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:01:04.048696 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:01:04.049061 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:01:04.049062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:01:04.049217 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:01:04.049273 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:04.049275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.050139 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.050140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:04.050940 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:01:04.050942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.051483 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.051484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:01:04.051555 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:04.051978 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:01:04.051983 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:01:04.053508 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:01:04.055795 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:04.055796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:04.055797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:04.056041 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:04.056255 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:01:04.056255 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:01:04.056256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:01:04.056310 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:01:04.056890 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:01:04.056891 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:01:04.056939 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 13:01:04.056987 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 13:01:04.057085 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 13:01:04.057974 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:04.048789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48040 (local address 127.24.83.60:40729)
0806 13:01:04.048830 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.048831 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.048839 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:01:04.048858 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.048859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.048867 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.048916 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.049363 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.050060 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.051050 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:04.051354 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:04.051413 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:04.053603 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:01:04.053604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:01:04.053604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:01:04.053611 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:01:04.055700 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.056131 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.056132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.056132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.056188 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:01:04.056381 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:01:04.056382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:01:04.056383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:01:04.056720 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:01:04.057581 (+ 861us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:04.057654 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 13:01:04.057798 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:01:04.059756 15806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c: Bootstrap starting.
I20260806 13:01:04.060535 15806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c: Neither blocks nor log segments found. Creating new log.
I20260806 13:01:04.060853 15806 log.cc:824] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c: Log is configured to *not* fsync() on all Append() calls
I20260806 13:01:04.062407 15806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c: No bootstrap required, opened a new log
I20260806 13:01:04.063972 15924 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:01:04.064682 15924 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:01:04.064657 15806 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } }
I20260806 13:01:04.064802 15806 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf2a91ea2f641f89fcaf6136f89820c, State: Initialized, Role: FOLLOWER
I20260806 13:01:04.064888 15806 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } }
I20260806 13:01:04.065269 15780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bca5cd66662c439c847bd1bc84f70d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cbf2a91ea2f641f89fcaf6136f89820c"
I20260806 13:01:04.065362 15779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:01:04.065249 15806 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:01:04.066236 15779 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [term 1 FOLLOWER]: Refusing update from remote peer bca5cd66662c439c847bd1bc84f70d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:01:04.065201 15934 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } } }
I20260806 13:01:04.066588 15934 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:04.067034 15780 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bca5cd66662c439c847bd1bc84f70d9b in term 1.
I20260806 13:01:04.067087 15904 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:01:04.069907 15934 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bca5cd66662c439c847bd1bc84f70d9b. Latest consensus state: current_term: 1 leader_uuid: "bca5cd66662c439c847bd1bc84f70d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } } }
I20260806 13:01:04.070016 15934 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:04.070207 15942 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bca5cd66662c439c847bd1bc84f70d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } } }
I20260806 13:01:04.070295 15942 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:01:04.071486 15951 catalog_manager.cc:1259] Loaded cluster ID: 1317dfe5df384bf7970e57b52a8471bb
I20260806 13:01:04.071574 15951 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c: loading cluster ID for follower catalog manager: success
I20260806 13:01:04.072798 15951 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c: acquiring CA information for follower catalog manager: success
W20260806 13:01:04.072991 15951 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260806 13:01:04.077771 15904 sys_catalog.cc:455] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bca5cd66662c439c847bd1bc84f70d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca5cd66662c439c847bd1bc84f70d9b" member_type: VOTER last_known_addr { host: "127.24.83.62" port: 38263 } } peers { permanent_uuid: "cbf2a91ea2f641f89fcaf6136f89820c" member_type: VOTER last_known_addr { host: "127.24.83.61" port: 32955 } } peers { permanent_uuid: "524c640dec07441cb9b7f93e1cc67a62" member_type: VOTER last_known_addr { host: "127.24.83.60" port: 40729 } } }
I20260806 13:01:04.077873 15904 sys_catalog.cc:458] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b [sys.catalog]: This master's current role is: LEADER
I20260806 13:01:04.079355 15924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bca5cd66662c439c847bd1bc84f70d9b: Generated new TSK 0
I20260806 13:01:04.079530 15924 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 13:01:04.111142 15888 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.109277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48054 (local address 127.24.83.60:40729)
0806 13:01:04.109328 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.109330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.110414 (+ 1084us) server_negotiation.cc:407] Connection header received
0806 13:01:04.110534 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.110535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.110547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.110579 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.111069 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48054: BlockingRecv error: recv got EOF from 127.0.0.1:48054 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 13:01:04.111985 15732 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.109718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51142 (local address 127.24.83.62:38263)
0806 13:01:04.109760 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.109762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.111410 (+ 1648us) server_negotiation.cc:407] Connection header received
0806 13:01:04.111516 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.111517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.111529 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.111559 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.111917 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51142: BlockingRecv error: recv got EOF from 127.0.0.1:51142 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 13:01:04.112768 15810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.110054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40498 (local address 127.24.83.61:32955)
0806 13:01:04.110094 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.110096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.112217 (+ 2121us) server_negotiation.cc:407] Connection header received
0806 13:01:04.112319 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.112320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.112332 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.112362 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.112706 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40498: BlockingRecv error: recv got EOF from 127.0.0.1:40498 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 13:01:04.115239 15810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.114467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40512 (local address 127.24.83.61:32955)
0806 13:01:04.114504 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.114505 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.114705 (+ 200us) server_negotiation.cc:407] Connection header received
0806 13:01:04.114815 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.114816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.114836 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.114867 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.115193 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40512: BlockingRecv error: recv got EOF from 127.0.0.1:40512 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 13:01:04.118476 15810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.117674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40518 (local address 127.24.83.61:32955)
0806 13:01:04.117709 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.117711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.117961 (+ 250us) server_negotiation.cc:407] Connection header received
0806 13:01:04.118062 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.118063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.118074 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.118103 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.118433 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40518: BlockingRecv error: recv got EOF from 127.0.0.1:40518 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 13:01:04.122359 15810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.121533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40530 (local address 127.24.83.61:32955)
0806 13:01:04.121567 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.121569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.121806 (+ 237us) server_negotiation.cc:407] Connection header received
0806 13:01:04.121905 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.121906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.121917 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.121947 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.122314 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40530: BlockingRecv error: recv got EOF from 127.0.0.1:40530 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 13:01:04.128420 15810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.127511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.24.83.61:32955)
0806 13:01:04.127548 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.127550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.127809 (+ 259us) server_negotiation.cc:407] Connection header received
0806 13:01:04.127955 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.127956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.127968 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.127999 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.128363 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40536: BlockingRecv error: recv got EOF from 127.0.0.1:40536 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 13:01:04.138666 15810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.137649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.24.83.61:32955)
0806 13:01:04.137692 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.137694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.137968 (+ 274us) server_negotiation.cc:407] Connection header received
0806 13:01:04.138072 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.138073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.138084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.138114 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.138595 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40552: BlockingRecv error: recv got EOF from 127.0.0.1:40552 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 13:01:04.157115 15810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.156109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.24.83.61:32955)
0806 13:01:04.156162 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.156165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.156484 (+ 319us) server_negotiation.cc:407] Connection header received
0806 13:01:04.156595 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.156596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.156609 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.156639 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.157053 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40556: BlockingRecv error: recv got EOF from 127.0.0.1:40556 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 13:01:04.191172 15810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.190399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.24.83.61:32955)
0806 13:01:04.190465 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.190467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.190602 (+ 135us) server_negotiation.cc:407] Connection header received
0806 13:01:04.190675 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.190676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.190689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.190756 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.191109 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40566: BlockingRecv error: recv got EOF from 127.0.0.1:40566 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 13:01:04.257356 15810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.256492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40576 (local address 127.24.83.61:32955)
0806 13:01:04.256576 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.256579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.256692 (+ 113us) server_negotiation.cc:407] Connection header received
0806 13:01:04.256818 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.256819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.256831 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.256912 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.257272 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40576: BlockingRecv error: recv got EOF from 127.0.0.1:40576 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 13:01:04.387245 15810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.386539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40592 (local address 127.24.83.61:32955)
0806 13:01:04.386635 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.386637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.386663 (+ 26us) server_negotiation.cc:407] Connection header received
0806 13:01:04.386748 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.386749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.386775 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.386829 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.387183 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40592: BlockingRecv error: recv got EOF from 127.0.0.1:40592 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 13:01:04.645140 15810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:01:04.644529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40594 (local address 127.24.83.61:32955)
0806 13:01:04.644605 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:04.644607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:04.644617 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:01:04.644646 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:04.644647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:04.644657 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:04.644709 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:01:04.645048 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40594: BlockingRecv error: recv got EOF from 127.0.0.1:40594 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 13:01:05.001461 15901 catalog_manager.cc:1259] Loaded cluster ID: 1317dfe5df384bf7970e57b52a8471bb
I20260806 13:01:05.001569 15901 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62: loading cluster ID for follower catalog manager: success
I20260806 13:01:05.002372 15901 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62: acquiring CA information for follower catalog manager: success
I20260806 13:01:05.002687 15901 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 524c640dec07441cb9b7f93e1cc67a62: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 13:01:05.073638 15951 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cbf2a91ea2f641f89fcaf6136f89820c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 13:01:05.161249 16023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:05.158371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40602 (local address 127.24.83.61:32955)
0806 13:01:05.158514 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:05.158516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:05.158576 (+ 60us) server_negotiation.cc:407] Connection header received
0806 13:01:05.158679 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:05.158680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:05.158750 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:05.158846 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:01:05.159009 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:05.159520 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:05.160392 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:05.160603 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:05.160684 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:05.160871 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:01:05.160989 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:01:05.161081 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:05.161120 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 13:01:05.161143 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260806 13:01:05.165355 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:05.163095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.24.83.60:40729)
0806 13:01:05.163212 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:05.163214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:05.163221 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:01:05.163259 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:05.163260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:05.163294 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:05.163360 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:01:05.163485 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:05.164009 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:05.164683 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:05.164870 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:05.164926 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:05.165053 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:01:05.165144 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:01:05.165197 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:05.165234 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 13:01:05.165250 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260806 13:01:05.170251 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:05.167115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51158 (local address 127.24.83.62:38263)
0806 13:01:05.167233 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:05.167234 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:05.167243 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:01:05.167301 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:05.167302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:05.167357 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:05.167448 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:01:05.167589 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:05.168427 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:05.169312 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:05.169611 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:05.169828 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:05.169904 (+ 76us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:01:05.170017 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:01:05.170077 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:05.170130 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 13:01:05.170155 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
I20260806 13:01:05.170344 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:05.167176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.24.83.60:40729)
0806 13:01:05.167210 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:05.167212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:05.167528 (+ 316us) server_negotiation.cc:407] Connection header received
0806 13:01:05.167555 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:05.167556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:05.167565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:05.167636 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:01:05.167789 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:05.168494 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:05.169499 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:05.169698 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:05.169773 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:05.169945 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:01:05.169975 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:01:05.170027 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:05.170225 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 13:01:05.170247 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:01:05.170388 16023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:01:05.167235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40614 (local address 127.24.83.61:32955)
0806 13:01:05.167281 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:01:05.167282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:01:05.167468 (+ 186us) server_negotiation.cc:407] Connection header received
0806 13:01:05.167549 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:01:05.167550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:01:05.167562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:01:05.167647 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:01:05.167817 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:05.168333 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:05.169398 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:01:05.169693 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:01:05.169778 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:01:05.170087 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:01:05.170121 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:01:05.170151 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 13:01:05.170267 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 13:01:05.170291 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:01:05.176404 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 15657
I20260806 13:01:05.182299 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 15737
I20260806 13:01:05.188009 24908 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzNDPsb/build/release/bin/kudu with pid 15813
2026-08-06T13:01:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2804 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5641 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (273851 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST